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PURPOSE 

On  November  4,  1992,  the  first  Workshop  on  Automation /Robotics 
for  Road  Construction , Maintenance,  and  Operations  was  sponsored 
by  the  Office  of  Advanced  Research  of  the  Federal  Highway 
Administration  (FHWA)  and  hosted  by  the  Robot  Systems  Division  of 
the  National  Institute  of  Standards  and  Technology  (NIST)  at 
Gaithersburg,  Maryland.  The  purpose  of  the  Workshop  was  to  bring 
together  experts  from  government,  industry,  and  academia, 
including  construction  contractors,  equipment  manufacturers, 
federal  and  state  highway  officials,  and  researchers,  to  discuss 
the  prospective  use  of  automation/robotics  in  road  construction, 
maintenance,  and  operations.  (A  list  of  attendees  can  be  found 
in  Appendix  A) . 

The  Workshop  was  to  serve  as  the  initial  gathering  in  a series  of 
year-long  events  to  inaugurate  the  new  FHWA  research  program  in 
automation  and  robotics.  It  was  a means  to  provide  the 
technology  developers  with  a reality  check  from  the  users,  the 
highway  construction,  maintenance,  and  operations  experts.  It 
was  an  initial  mechanism  to  examine  the  requirements  and 
opportunities  for  automation/robotics  in  highway  systems. 

The  purpose  of  this  report  is  to  document  the  workshop,  including 
presentations,  participant  discussions,  and  the  results  of 
working  group  deliberations.  A videotape  of  the  Workshop  is 
available  at  NIST. 


BACKGROUND 

Automation  technology  promises  to  increase  safety  and 
productivity  while  reducing  costs  and  the  adverse  impact  of  work 
sites  on  traffic.  The  new  FHWA  automation/robotics  program 
emphasizes  finding  near-term  solutions  to  pressing  problems  and 
fielding  useful  equipment. 

In  recent  years  there  has  been  significant  progress  in  the 
development  of  Government  and  commercial  software  and  hardware 
systems  for  various  automation  applications.  Advanced  sensor 
systems  for  inspection,  automated  databases  and  program 
management  software,  and  robotic  vehicles  are  some  of  the  new 
automation  technology  which  promises  to  benefit  road 
construction,  maintenance,  and  repair. 

For  example,  robotics  for  military  applications,  including  the 
development  robotic  excavators,  dozers,  and  inspection  vehicles 
with  various  kinds  of  sensors,  can  be  modified  for  road 
construction  applications.  Robotic  systems  have  also  been 
developed  for  coal  mining,  hazardous  waste  cleanup,  and  building 
construction.  In  1990,  the  California  Department  of 
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Transportation  (CALTRANS)  initiated  the  Automated  Highway 
Maintenance  and  Construction  Technology  Program;  the  Japanese 
have  had  active  programs  for  nearly  ten  years. 

Automation  of  road  construction,  maintenance,  and  operations  can 
increase  safety  (highway  workers  suffer  more  than  5,000  injuries 
and  nearly  800  deaths  per  year) , productivity,  and  quality,  while 
reducing  costs  and  the  adverse  impact  of  work  sites  on  traffic. 

Potential  functions  include: 

Construction : 

* Clearing/ sediment  control 

* Earthwork 

* Shoring 

* Paving 

* Marine 

* Bridges 

* Utilities 

* Signing 

* Rehabilitation 

Maintenance: 

* Concrete  patching 

* Repaving 

* Resigning 

* Bridge/tunnel  repair 

* Plowing/ sweeping 

* Painting/striping 

* Mowing 

* Inspection  of  roads  and  bridges 

Operations : 

* Special  traffic  control  for  construction 

* Highway  system  management 

* Safety  management 

* Incident  management 

CALTRANS,  for  example,  has  been  developing  automation  systems  for 
road  maintenance,  including:  automated  pavement  marking, 
automated  paint  striping,  automated  litter  collection,  and 
automated  crack  sealing. 

Other  near-term  robotics  projects  under  discussion  include:  a 
flag  robot  for  temporary  traffic  control;  a robot  for  painting 
road  letters;  a robot  for  placing  and  removing  traffic  cones;  a 
robot  for  constructing  sound  isolation  walls;  a robot  for 
clearing  hazardous  material  after  accidents. 
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Automation  could  connect  sensors  and  equipment  on  a job  site  with 
project  databases  and  management  software.  For  example,  as  an 
automated  excavator  performs  its  job,  it  could  report  its 
movements  and  accomplishments  to  a database  which,  in  turn, 
automatically  updates  project  progress  records  and  revises,  if 
necessary,  the  work  plan.  Inspection  of  road  surfaces  and 
bridges  could  be  accomplished  automatically  with,  video  and  other 
sensors,  using  automated  sensor  processing  techniques.  The 
results  of  the  inspection  would  be  organized  and  entered  into  a 
database  automatically  as  the  inspection  took  place.  The 
inspection  itself  could  be  accomplished  using  a robotic  vehicle. 

There  has  been  research  and  development  in  applications  areas 
relevant  to  road  construction,  maintenance  and  operations,  but 
much  of  the  work  focuses  on  building  construction,  or  military 
applications,  or  coal  mining  (pertinent  literature,  now  at  NIST, 
is  listed  in  Appendix  B) . A more  focused  effort  on  highway 
applications  is  needed,  so  the  Transportation  Research  Board  is 
offering  their  first  sessions  dedicated  to  road  construction 
robotics  and  automation  in  their  72nd  annual  meeting  (January 
1993).  Examples  of  previous  work  on  automation/robotics  for  road 
and  bridge  construction,  maintenance,  and  operations  include: 

* Expert  System  for  Management  of  Low  Volume  Flexible  Pavements 

* Artificial  Intelligence  To  Locate  And  Repair  Potholes 

* Automated  Surveying 

* Automated  Position  And  Control  Systems  Using  Lasers  And 
Electromagnetic  Signals 

* Bar  Code  Applications  in  Construction 

* Robotic  Excavation 

* Artificial  Intelligence  And  Computer  Simulation  To  Plan  And 
Control  Earthmoving  Operations 

* Computer  Aided  Rigging  Design  System 

* Object-Oriented  Programming  In  Robotics  Research  For  Excavation 

* Pavement  Distress  Video  Imager  To  Quantify  Pavement  Cracking 
From  Video  Images 

* An  Expert  System  For  Optimal  Tower  Crane  Selection  And 
Placement 

* A Knowledge-Based  Approach  To  Construction  Coordination 

* An  Object-Oriented  Simulation  System  for  Construction  Process 
Planning 

* Intelligent  Database  Applications  On  Signal  Maintenance 
Activities 

* Knowledge  Representation  for  Fatigue  Evaluation 

* A Computer  Assisted  System  For  Construction  Robot 
Implementation  Logistics 

* Knowledge  Elicitation  Techniques  For  Construction  Scheduling 

* Expert  Systems  for  Bridge  Monitoring 

* Adaptive  Control  for  Robotic  Rebar  Bending 

* Construction  Schedule  Generation  Using  AI  Tools 

* Automated  Pavement  Surface  Distress  Evaluation 
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* Computer  Analysis  of  Segmentally  Erected  Prestressed  Concrete 
Bridges 

* An  Expert  System  For  Design  And  Analysis  Of  Highway  Bridges 

* Investigation  Of  The  Bridge  Vehicle/ Superstructure  Interaction 
Problem  Via  Computer-Based  Methodology 

* Database  Design  For  Seismic  Evaluation  Of  The  San  Francisco  Bay 
Bridge 

* A Construction  Expert  System  For  The  Preliminary  Design  Of 
Reinforced  Concrete  Structures 

* Automatic  Pavement-Distress-Survey  System 

* Highway  Pavement  Surfaces  Reconstruction  by  Moire 
Interferometry 

* A Field  Prototype  of  a Robotic  Pavement  Crack  Sealing  System 

* A Design  for  Automated  Pavement  Crack  Sealing 

* Integration  of  Diverse  Technologies  for  Pavement  Sensing 

* Force  Feedback  Excavator  and  Material  Handling  System 

* Automated  Pavement  Crack  Filler 

* Perception  and  Control  for  Automated  Pavement  Crack  Sealing 

* Adaptive  Control  for  Robotic  Backhoe  Excavation 

* Subsurface  Pavement  Structure  Inventory  Using  Ground 
Penetrating  Radar  and  a Bore  Hole  Camera 

* A Data  Base  Program  for  Preparing  and  Reporting  Concrete  Mix 
Designs 

* Pavement  Image  Processing  Using  Neural  Networks 

* An  Expert  System  For  Construction  Contract  Claims 

* Analysis  and  Generation  of  Pavement  Distress  Images  Using 
Fractals 

* Measuring  Highway  Inventory  Features  Using  Stereoscopic  Imaging 
System 

* Using  Geographic  Information  Systems  For  Highway  Maintenance 

* Simulation  for  Construction  Planning  and  Control 

* Automated  Bridge  Plans  By  Computer  Aided  Software 

* Electronic  Communication  Between  Project  Participants 

* CAD-Integrated  Rebar  Bending 

* Knowledge  Based  Expert  System  for  Construction  Scheduling 

* Pavement  Design  Using  An  Expert  System 

* A Database  Approach  for  CAD/KBES  Integration  for  Construction 
Planning 

* Real-Time  Project  Tracking 

* A Relational  Database  For  Long-Span  Highway  Bridges 

* A Construction  Information  Management  System 

* Surface  Condition  Expert  System  For  Pavement  Rehabilitation 
Planning 

* Digital  Imaging  Concepts  And  Applications  In  Pavement 
Management 

* An  Expert  System  for  Pavement  Rehabilitation  Decision  Making 

* An  Expert  System  for  Contractor  Prequalification 

* Generic  Framework  For  Evaluation  Of  Multiple  Construction 
Robots 

* Framework  for  Construction  Robot  Fleet  Management  System 

* A Pavement  Management  Information  System  for  Evaluating 
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Pavements  and  Setting  Priorities  for  Maintenance 

* Autonomous  Robot  Excavator 

* Integrating  Data  Bases  For  Executing  Automated  Construction 
Tasks 

* Computer  Integration  For  Automated  And  Flexible  Construction 
Systems 

* Integrating  Voice  Recognition  Systems  with  the  Collection  of 
Project  Control  Data 

* Sensors  And  Expert  Systems  In  Production  Optimization  Of 
Earthmoving  Scrapers 

* Control  System  Architecture  for  Unmanned  Ground  Vehicles 

* Teleoperated  Excavator 

* A Graphical  Interface  For  Curved  Steel  Girder  Bridges 

* An  Expert  System  for  Diagnosing  and  Repairing  Cracks  in  Cast- 
in-Place  Concrete  Structures 

* Probabilistic  Scheduling  in  Tunneling 

* A Database  For  Tunnel  Planning  And  Estimating 

* Kinematics  and  Trajectory  Planning  for  Robotic  Excavation 

* Design  Considerations  For  Automated  Crack  sealing  Machinery 

* A Computer  System  For  Highway  Bridge  Rating  And  Fatigue  Life 
Analysis 

* Knowledge-Based  Construction  Scheduling 

* A Hypertext  Database  for  Asphalt  Paving 

* A Knowledge-Based  Expert  System  for  Quality  Assurance  of 
Concrete 

* New  Capability  for  Remote  Controlled  Excavation 

But  it  is  really  up  to  the  users  of  the  new  technology  (the 
highway  contractors  and  departments)  to  tell  the  developers  (the 
automation  experts)  what  they  really  want  - what  will  make  a 
crucial  difference  on  the  job  site.  And  the  automation  eguipment 
must  be  practical  - it  must  do  the  job  better,  faster,  or 
cheaper. 


A PRECIS  OF  THE  WORKSHOP 

The  following  description  of  the  Workshop  paraphrases  the  various 
presenters  and  participants  in  an  attempt  to  summarize  the  key 
issues  discussed. 


Dr.  Ernie  Kent 

Dr.  Ernie  Kent,  of  the  Robot  Systems  Division  of  NIST,  welcomed 
the  representatives  of  the  highway  and  scientific  communities 
attending  the  Workshop.  The  goal  of  the  Workshop,  he  stated,  was 
to  allow  the  technical  community  to  learn  what  are  the  important 
issues  from  the  perspective  of  the  highway  community.  The 
principal  work  of  the  day  will  fall  on  the  highway  community. 
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Dr.  Kent  introduced  Bob  Finkelstein,  of  Robotic  Technology  Inc., 
who  helped  organize  the  Workshop.  All  problems  and  complaints 
should  be  directed  at  Bob,  suggested  Dr.  Kent. 

Dr,  Kent  then  introduced  the  moderator  for  the  Workshop,  Dr. 
Richard  Wright,  Director  of  the  Building  and  Fire  Research 
Laboratory  at  NIST,  Dr.  Charles  Woo,  Research  Manager  For 
Robotics  at  the  Office  of  Advanced  Research  of  the  FHWA,  sponsor 
of  the  Workshop,  and  Mr.  Tom  Pasko,  Director  of  the  Office  of 
Advanced  Research  of  the  FHWA. 


Dr. Charles  Woo 

Dr.  Woo,  in  his  introductory  remarks,  noted  that  automation,  in 
the  context  of  interest,  should  be  all-inclusive,  including  the 
entire  life-cycle  of  the  highway  system.  Robotics  technology 
promises  to  improve  safety  and  increase  productivity,  and  the 
technology  is  sufficiently  established  to  warrant  serious 
consideration  for  use  in  highway  construction,  maintenance,  and 
operations . 

Robotics  can  be  used  in  all  phases  of  highway  construction: 
production  of  highway  materials,  construction  of  highways 
(including  quality  control) , highway  maintenance  and  operations 
(including  inspection  and  monitoring)  - especially  in 
environments  which  are  hazardous  or  difficult  to  access. 

Thus  far,  use  of  this  technology  in  highway  transportation  has 
been  limited  due  to  a lack  of  understanding  of  the  technology  and 
a lack  of  overall  research  planning  for  applications  of  this 
technology  to  highway  transportation.  Therefore,  the  FHWA  is 
sponsoring  a study  to  be  conducted  by  NIST,  an  in-depth 
evaluation  of  the  feasibility  of  robotics  to  highway 
construction,  maintenance,  and  operations.  It  will  include  an 
assessment  of  the  current  technology,  as  well  as  the  development 
of  new  highway  improvements  and  methods.  The  study  does  not 
include  robotics  applications  to  the  Intelligent  Vehicle  Highway 
System  (IVHS) . The  DoT  IVHS  program  has  its  own  robotics 
applications  research. 

The  first  step  in  this  study  is  to  hold  this  one-day  Workshop  of 
knowledgeable  industry  experts,  construction  contractors,  and 
equipment  manufacturers,  for  a strategic  plan  of  research.  We 
look  forward  to  a frank  exchange  of  opinions  and  ideas  by  all 
participants . 


Dr.  Richard  Wright 

The  procedure  for  the  Workshop  is: 
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(1)  There  will  be  a few  words  about  the  objectives  and  how  they 
were  organized. 

(2)  Dr.  Leonhard  Bernold  will  give  his  view  of  the  state-of-the- 
art  of  automation  and  robotics  technology  - a view  to  be  used  in 
subsequent  discussions  to  describe  how  these  technical  abilities 
can  be  directed  to  support  the  needs  of  highway  engineering 
through  the  whole  life  cycle  (design,  construction,  operation, 
maintenance,  and  repair.  The  FHWA  wants  to  understand  - and  we 
want  to  understand  - from  the  users  how  robotics  and  automation 
can  respond  to  the  needs  of  the  highway  community. 

(3)  We  will  have  a plenary  issues-raising  session,  to  look  at 
candidate  issues  to  be  explored  for  the  effective  application  of 
automation  and  robotics. 

(4)  We  will  separate  into  working  groups,  which  will  use  their 
judgement  and  information  from  the  plenary  session,  to  point  out 
what  they  think  are  the  issues  and  how  the  issues  should  be 
addressed  by  the  FHWA  and  highway  community  working  together. 

(5)  The  working  groups  will  then  report  to  the  plenary  session, 
and  everyone  will  seek  to  synthesize,  out  of  the  five  views  of 
the  five  working  groups,  some  reasonable  consensus  as  to  the  most 
important  issues  to  be  explored  for  bringing  automation  and 
robotics  into  effective  use  in  the  highway  system. 

The  Workshop  is  being  videotaped,  but  opinions  will  not  be 
ascribed  to  specific  individuals  in  order  to  encourage 
participants  to  be  candid. 


Goals  And  Objectives 

It  is  not  only  the  hardware  moving  things  on  the  work  site  that 
is  important,  but  also  the  information  management  that  takes 
place  throughout  the  whole  highway  process. 

A previous  workshop  six  years  ago  at  NIST,  concerning  automation 
in  construction  in  large  scale  assemble,  was  similar  in  purpose 
to  this  one  but  focused  on  shipbuilding,  building  construction, 
aircraft  manufacturing  and  the  like.  The  technology  that  emerged 
as  the  top  priority  in  that  workshop  was  a site  positioning 
system  - a system  that  would  automatically  record,  in  real-time, 
on  the  site,  what  is  there  and  where  is  it.  It  came  to  the  top 
of  the  list  because  we  need  this  system  whether  or  not  there  is  a 
robot  on  the  site.  Likewise,  a highway  engineer  would  find  it 
valuable  to  know,  any  time  day  or  night,  what  is  on  the  site, 
what  condition  it  is  in,  and  where  it  is  being  stored.  The 
Construction  Industry  Institute  and  other  organizations  are  now 
working  to  bring  this  technology  to  reality. 
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The  objectives  of  this  Workshop  (please  see  Appendix  C)  are  to: 

* Define  the  functional  needs  that  can  be  met  by  automation  and 
robotics,  such  as  obvious  needs  for  safety. 

* Define  technological  developments  required  to  meet  these  needs. 
Some  technology  is  available  off-the-shelf  and  some  is  not  yet 
ready.  Highway  engineering  is  not  a low-tech  industry,  and  it 
has  challenging  informational  requirements. 

* Define  organizational  and  institutional  changes  needed  to 
exploit  automation  and  robotics  for  highways.  Institutional  and 
organizational  problems  must  be  addressed  before  we  can  take 
advantage  of  the  capabilities  of  automation  in  the  highway 
enterprise.  That  is  why  it  is  important  to  have  highway  system 
people  here  to  address  institutional  barriers,  which  are  just  as 
important  as  the  technical  barriers,  and  why  we  need  to  start 
early  on  overcoming  them.  We  need  to  be  sensitive  to  these  kinds 
of  issues  - and  to  legal  issues  as  well. 

In  the  typical  progression  of  innovation,  a process  which  is 
relevant  to  the  introduction  of  new  technology  to  the  highway 
system,  there  are  three  major  steps: 

(1)  An  existing  product  or  process  is  replaced  with  innovation. 

(2)  A product  or  process  is  modified  to  exploit  the  potential  of 
innovation. 

(3)  Roles  and  responsibilities  are  modified  to  fit  the  new 
environment . 

For  example,  we  can  replace  the  flagman  with  a robot,  where  the 
immediate  advantage  is  that  the  robot  can  be  hit  by  a car  but 
leave  no  widow.  This  type  of  substitution  is  only  the  first  step 
in  innovation  because  once  you  get  new  technology  you  want  to 
change  products  or  processes  to  fully  exploit  the  new  technology. 
For  example,  welding  replaced  rivets,  but  the  connections  still 
looked  as  if  they  were  to  be  riveted  or  bolted.  But  after  people 
got  comfortable  with  the  new  technology,  the  nature  and  shapes  of 
the  connections  were  changed  to  take  advantage  of  the  welding 
process.  Another  example:  automobiles  were,  at  first,  really 
horseless  carriages. 

Hopefully,  in  addition  to  seeing  how  we  can  replace  existing 
equipment  with  robotics  and  automation,  we  can  get  second 
thoughts  about  how  to  change  whole  procedures  and  products  to 
take  advantage  of  robotics  technologies. 

We  may  not  get  to  it  this  meeting,  but  when  we  introduce  new 
technologies,  and  modify  products  and  processes  to  exploit  these 
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technologies,  we  begin  to  see  that  organizations,  roles,  and 
responsibilities  have  to  change  to  deal  with  the  technologies. 

For  example,  in  highway  bridge  engineering,  we  have  gone  beyond 
the  point  where  an  engineer  designs  the  bridge  and  a contractor 
builds  it.  The  contractor  now  introduces  a new  design  and  the 
final  product  is  a result  of  synthesis.  With  improved 
information  flow  and  technigues  for  the  design,  construction, 
operation,  and  maintenance  of  highways,  we  will  see  changes  in 
the  way  highway  systems  are  procured  and  managed.  This  is  a bit 
beyond  our  charter  for  today,  but  we  should  keep  it  in  mind 
because  there  are  important  issues  here  for  the  FHWA  to  attend  to 
early  in  the  program. 


Dr.  Leonhard  Bernold 

It  is  impossible  to  present  the  state-of-the-art  of  robotics  in 
the  time  available.  The  goal  is  to  share  some  of  the  important 
lessons  learned,  to  present  a framework  with  a global,  rather 
than  local,  view.  We  will  highlight  some  of  the  needs 
identified,  highlight  some  basic  concepts,  and  provide  a common 
sense  approach,  mixed  with  some  far-out  ideas  because  it  doesn't 
hurt  to  dream  (please  see  the  Charts  in  Appendix  D) . 

There  are  many  definitions  of  automation,  but  it  doesn't  really 
matter  which  one  picks.  One  motivation  to  automate  is  safety, 
but  what  is  the  broad  issue?  Back  injury  is  one  of  the  most 
crucial,  costly  safety  issues.  By  far  back  injuries  are  greatest 
percentage  of  injuries  (over  24%  - the  next  highest,  finger  or 
knee  injuries,  are  at  about  7%) , and  the  cost  is  horrendous.  The 
average  weight  of  cement  bags  is  94  pounds,  while  a man  can  lift 
60-65  pounds  on  a regular  basis.  Technology  can  be  used  so 
people  do  not  have  to  lift,  as  in  Germany  where  there  is  a 
mechanical  slave  that  moves  bricks  for  workers. 

Automation,  in  addition  to  providing  safe  and  humane  workplaces, 
reducing  waste,  supporting  workers,  providing  consistently  high 
guality,  and  increasing  productivity,  can  provide  other  benefits 
to  mankind,  such  as  aesthetics  - beauty  and  art.  Robots  can,  for 
example,  make  stone  mosaics  more  affordable  so  that  can  be  used 
more  widely. 

But  automating  a mess  just  leads  to  an  automated  mess.  Processes 
and  facilities  need  to  be  reorganized  to  accommodate  automation, 
and  just  the  reorganization  itself  can  increase  productivity  by 
improving  the  flow  of  materials  and  reducing  waste  - before  any 
robots  are  actually  introduced.  As  an  example  of  a mess,  just 
consider  a typical  construction  site  where  rebars  are  scattered 
all  over  the  site. 

There  are  several  steps  in  the  life-cycle,  of  data  and  objects  of 
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construction  which  are  amenable  to  automation:  need;  concept 
studies;  design/engineering;  planning/detailing;  procurement; 
construction;  operation;  rehabilitation;  operation;  and 
demolition.  Streamlined  material  and  data  flows  can  be  achieved 
with  enhanced  electronic  and  mechanical  systems  having  suitable 
computer  interfaces.  For  example,  rebar  bending,  which  is 
currently  a manual  process,  can  be  automated  on  a conventional 
rebar  bending  table  with  a computer-controlled  system  using 
sensory  feedback  from  the  bending  process  and  motor  control  of 
the  bending  process.  But  this  system  can  be  expanded  to  automate 
process  planning,  to  include  information  on  where  and  when  the 
bars  are  needed.  We  need  to  look  at  automating  the  entire 
system,  not  just  the  bending  of  the  bar. 

With  computer-aided  design  and  drafting,  and  automated  process 
planning,  we  can  eliminate  some  activities  and  consequent  delays 
in  the  electronic  data  flow  in  tasks  preceding  construction, 
operation,  and  maintenance.  At  the  construction  site,  where 
there  are  robotic  excavators  and  the  like,  robotic  systems  should 
be  linked  to  the  data  stream  to  guide  the  work.  Physical  systems 
(robotic  excavators  and  dozers,  for  example)  and  processes  should 
be  linked  together  and  the  data  stream:  this  is  the  dream. 

Large  robotic  systems,  like  excavators  (which  are  being  developed 
in  Japan) , can  work  accurately  and  precisely  using  site 
measurement  and  position  techniques,  such  as  provided  by  laser 
positioning  systems.  There  is  immediate  feedback  from  the 
excavator  as  to  where  it  is  and  what  it  is  doing.  Such  systems 
can  get  people  out  of  ditches,  which  is  a very  dangerous 
environment,  and  increase  productivity.  NIST,  for  example,  has 
the  SPIDER  robotic  crane  which  can  be  used  in  road  construction. 

There  are  also  simple  things  which  can  be  automated  to  support 
the  worker,  such  as  a smart  nailer.  It  incorporates  a stud 
sensor  to  allow  the  nailer  to  find  the  stud  and  drive  the  nail 
into  it.  It  allows  an  inexperienced  worker  to  perform  as  if  he 
were  a skilled  craftsman.  Skilled  craftsmen  do  not  benefit  much 
from  such  technology,  but  less  skilled  workers  improve 
considerably. 

Bar  codes  are  a tremendous  technology  with  lots  of  opportunities 
in  road  construction,  including  data  entry,  tracking  of 
materials,  and  bridge  maintenance.  Speech  recognition  and  pen- 
based  computing,  are  useful  automation  tools  for  people  who  don't 
like  to  type  or  write,  like  some  construction  foremen. 

Prototyping  is  very  important.  Get  prototypes  to  the  field  to 
serve  as  catalysts,  especially  for  people  who  are  reluctant  to 
change,  and  to  generate  synergy  among  different  individuals  and 
groups . 
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To  achieve  competence,  commitment,  and  effectiveness  in  the 
introduction  of  automation  and  robotics  for  highway  applications, 
government,  industry,  and  academia  must  work  together,  perhaps 
for  10,  20,  or  30  years.  Government  must  provide  leadership.  In 
academia,  students,  excited  by  the  technology,  can  work  with 
industry  performing  useful  research  with  practical  results  based 
on  actual  field  experience. 

In  summary: 

* Automation  should  start  with  cleaning  up  the  mess. 

* Automation  should  be  based  on  a global,  rather  than  a local, 
view. 

* An  electronic  data  trail  can  serve  as  a direct  thread  for 
thinking  globally. 

* Multi-disciplined,  multi-organizational  technology  and 
processes  should  be  emphasized. 

* Many  inexpensive  prototypes  and  field  types  should  be  employed 
to  get  the  technology  rapidly  into  the  hands  of  the  users,  to  get 
them  excited  and  energized. 


Mr . Thomas  Pasko 

The  Office  of  Advanced  Research  (please  see  Appendix  E)  is 
interested  in  making  big  improvements,  with  respect  to  the 
infrastructure,  in  productivity  and  effectiveness.  For  example, 
in  the  late  1950s  we  switched  from  the  old  form  riding  concrete 
placement  equipment,  which  had  a paving  crew  of  about  100  people. 
We  developed  the  slip  form  paver  and  the  paving  crew  was  reduced 
to  about  25  people.  Now  we  would  like  to  reduce  the  25  people  to 
10,  if  we  can  do  it. 

With  highways,  we  have  a disaggregated  market.  There  is  an 
incorrect  impression  that  the  FHWA  owns  all  the  roads  and  order 
the  use  of  new  technology.  It  doesn't  work  that  way.  FHWA  owns 
6%  of  the  roads,  like  the  GW  Parkway  and  Dulles  Access  Road.  The 
FHWA  works  with  the  states,  which  own  23%  of  the  roads.  The 
remaining  71%  is  owned  by  39,000  local  units  (such  as  counties, 
municipalities,  and  townships) . This  disaggregated  market  must 
be  sold  on  the  technology  as  well  - 39,000  units,  plus  the 
district  offices  of  the  states,  plus  the  regional  offices  of  the 
FHWA.  It  is  very  difficult  to  get  anything  new  into  practice. 

The  technology  is  very  traditional  and  brute-force  oriented. 

There  are  about  600,000  people  in  the  various  highway  departments 
and  FHWA,  plus  39,000  organizations  as  well. 
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There  is  a federal  aid  system  which  gets  funding  into  22%  of  the 
highways,  so  78%  have  no  federal  money  at  all.  So  FHWA's 
influence  is  limited.  (But  the  22%  handle  80%  of  the  traffic  - 
federal  aid  roads  are  high-value) . FHWA  works  primarily  through 
a federal  aid  partnership:  federal  funding  goes  to  states,  which 
then  match  the  funding.  The  result  is  like  having  50  countries 
doing  their  own  thing. 

The  FHWA  encourages  research  and  tries  to  get  people  to  use  it. 
Often  they  will  use  new  technology  when  there  are  financial 
incentives,  and  then  revert  back  to  what  they  are  comfortable 
with  when  the  incentives  are  gone. 

The  Office  of  Advanced  Research  includes  the  areas  of:  national 
service  center;  decision  analysis  in  transportation;  energy 
conservation  related  technology;  high  performance  materials; 
self-monitoring  systems;  computer  driven  technologies;  and 
robots/automation/man-machines.  The  latter  includes  the  use  of 
robots  and  automation  for  production,  construction,  maintenance, 
continuous  quality  control,  and  hazardous  environments. 

As  an  indication  of  what  the  states  think,  consider  one  recent, 
important  report,  from  Purdue  University,  on  the  use  of  advanced 
technologies  in  the  Indiana  Department  of  Transportation.  It 
noted  that  very  few  states  actually  have  projects  in  robotics. 

It  is  getting  very  little  emphasis,  except  in  California.  Most 
of  the  research  and  development  is  in  computerized  design, 
analysis,  and  planning;  database  management  and  information 
systems;  and  highway  traffic  operations  and  management.  In  terms 
of  the  actual  use  of  computer  technology  in  State  DoTs  (as  of 
1985) , there  was  no  robotics  or  artificial  intelligence  in  use. 
Database  management  topped  the  list  of  computer  use. 

There  are  many  possible  barriers  to  the  adoption  of  advanced 
technology.  Key  barriers,  as  determined  by  survey,  are:  high 
initial  cost;  lack  of  trained  personnel;  high  operation  and 
maintenance  cost;  general  resistance  to  change;  uncertainty  about 
potential  benefits;  and  uncertainty  about  the  type  of 
technologies  that  can  be  used. 

As  an  egregious  example  of  a hazardous  job  which  might  be 
automated,  and  which  was  photographed,  consider  a worker  who  is 
sitting  on  the  skids  of  a hovering  helicopter  repairing  an 
electrical  high-tension  wire. 

Unmanned,  miniature  helicopters  have  been  used  by  the  FHWA  to 
inspect  bridges  (and  CALTRANS  has  a similar  project) . The 
military  has  advanced,  high-tech  unmanned  air  vehicles  (UAVs) 
while  the  FHWA  has  low-tech  ones.  A technology  gap  exists.  We 
must  change  paradigms  and  procedures  to  adopt  military  systems 
for  highway  use. 
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We  can  create  "pull"  for  technology.  We  want  to  have 
intergovernmental  activities,  like  with  NIST,  to  create 
synergies,  to  use  new  technology,  such  as  automation  and 
robotics,  in  areas  that  the  government  controls.  We  can  get 
around  building  codes  and  liability  issues,  and  the  like,  by 
using  new  technology  in  government-owned  facilities.  The  U.S. 
government  owns  230,000  miles  of  highway,  supports  270,000 
bridges,  owns  417,000  buildings,  etc.  We  can  do  many 
experimental  projects  on  government  facilities,  and  many  products 
can  be  made  in  government  facilities. 


Brainstorming  Session  With  Dr.  Richard  Wright 

We  would  like  to  generate  ideas  for  consideration  in  the  working 
groups.  As  a starting  point  for  discussions,  we  will  first 
consider  needs  or .requirements  for  highway  systems  performance, 
followed  by  the  opportunities  for  automation/robotics  in  highway 
systems . 

Requirements  for  highway  system  performance  might  consist  of: 

* Economy  (in  terms  of  life  cycle  perspective,  not  just  first 
costs) 

* Functionality  (how  well  do  they  really  serve  the  needs  of  the 
users  of  the  highway  system?) 

* Durability  (do  they  maintain  their  initial  properties 
effectively  over  time?) 

* Time  Savings  (time  is  money,  so  reduce  the  times  for  design, 
construction,  operations,  and  maintenance) 

* Safety  (most  likely  to  gain  substantial  attention) 

* Environment  (the  system  should  be  environmentally  benign) 

* Regulatory  Compliance  (look  at  ways  robotics/automation  can 
reduce  the  burden  of  dealing  with  regulations) 

What  are  the  requirements  from  the  point  of  view  of  state  and 
local  governments,  FHWA,  materials  suppliers,  contractors,  etc.? 
Which  are  the  most  important?  Which  are  easiest  to  do?  Most 
important  to  CALTRANS  is  safety  - to  get  workers  off  the  road. 

Opportunities  in  automation/robotics  include: 

* Measurement  and  site  data  acquisition 

* Automation  of  quality  assurance  and  inspection 
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* Materials  handling  and  management 

* Earth  moving,  fabrication,  placement,  finishing 

* integrated  project  information  systems 

* Project  management 

* Operations,  maintenance,  repair 

The  Workshop  will  separate  into  five  heterogenous  groups.  The 
objectives  of  the  groups  will  be  to  determine:  on  which 
priorities,  with  respect  to  road  construction,  maintenance,  and 
operation,  automation/robotics  should  focus,  and  what  are  the 
most  important  issues.  Each  group  will  report  its  findings  to 
the  rest  of  the  Workshop. 

Before  separating  into  groups,  the  plenary  session  will  first 
generate  an  initial  list  of  the  most  important  requirements  or 
needs  for  the  groups  to  consider: 

* Safety 

* Durability  (which  is  related  to  safety  - do  it  carefully  and  it 
lasts  a long  time) 

* Simple  to  regulate  (so  contractors  will  use  them) 

* Performance-based  requirements  (define  system  requirements  to 
use  automation/robotics  as  a given;  characterize  user's  needs) 

* Open  systems 

* Able  to  define  clear  economic  benefits  (performance-related 
specifications;  efficient  procedures  by  contractors;  better 
materials) 

* Evolutionary,  not  revolutionary,  approach  (near-term  usability; 
no  esoteric  systems) 

* New  approaches  (such  as  high-pressure  water  instead  of 
backhoes) 

* Labor  considerations  (education/ training  in  use  of  the 
technology;  potential  job  losses) 

* Life-cycle  cost/benefits 

* Political,  as  well  as  institutional  and  structural,  barriers  to 
acceptance  of  new  technology 
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* Minimum  initial  cost  (lowest  initial  cost  drives  new  programs, 
not  life  cycle  costs) 

* Jobs 

* Labor  costs/quality 

* Workers  want  better  equipment,  better  man/machine  interface 
(more  done  with  less  effort;  systems  must  work  well  with  real 
people) 

* Leverage  DoE,  DoD,  European,  and  Japan  robotics  programs 

* Education/retraining  (for  enhancing  labor  pool) 

* Equipment  maintenance  and  repair  (requirements  for  new  systems) 

* Demonstrate  to  contractor  he  can  make  money  - give  confidence 
in  technology  (can't  be  just  marginally  beneficial) 

* Investment  in  R&D/technology  by  industry  and  government 
(require  that  advanced  technology  be  used  on  projects;  carrot 
better  than  stick?;  cost  sharing  of  technology  demonstrations; 
encourage  innovation  with  procurement  of  systems) 

* Establish  a network  of  stakeholders  for  this  technology 

* Need  to  reduce  risk  to  initial  users  of  new  technology  (capital 
risk;  liability  risk;  how  to  spread  risk  equitably) 

* Determine  what  makes  the  technology  economic  and  attractive  to 
users  (such  as  improving  the  performance  or  cost  of  new  workers; 
new  people  needed  in  the  trade  who  can  do  a good  job;  contractors 
are  concerned  with  near-term  costs  because  they  need  to  stay  in 
business) 

* Requirements  to  force  use  of  robotic  technology  - driving 
forces  (specifications  which  imply  the  use  of  certain  technology, 
as  in  Japan  where  companies  must  show  they  invest  in  R&D  in  order 
to  get  contracts) 

* Historical  drivers  are  cost,  safety,  and  health 

* Not  Invented  Here  (NIH)  syndrome  must  be  overcome 

* Perhaps  the  U.S.  should  adopt  European  model:  non-adversarial 
relationships;  government,  industry,  and  academia;  partnering 
(but  there  are  questions  of  scale  - the  U.S.  has  smaller 
construction  companies  than  in  Europe) 

The  plenary  session  will  now  generate  a list  of  potential 
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opportunities,  with  respect  to  the  introduction  of 

robotics /automation  for  highway  applications,  for  the  working 

groups  to  consider: 

* Use  incentives  (such  as  buy-back  incentives  where  the 
government  buys  back  old  equipment,  and  subsidies  for  labor  or 
equipment) 

* Reduce  hazards,  which  leads  to  savings  in  labor  costs  (working 
in  trenches  is  hazardous;  and  falling  is  the  main  lethal  accident 
in  construction) 

* Employ  grade  control  (using  automated  measurement;  materials 
oriented) 

* Eliminate  rollers  and  roller  operators:  compact  behind  the 
machine  automatically  in  one  pass  instead  of  two  or  three; 
eliminate  at  least  one  roller  operator) 

* Eliminate  handwork,  such  as  raking,  which  leads  to  injuries 

* Use  quick  methods  of  maintenance,  such  as  patching,  pavement 
repairs,  and  bridge  painting  to  reduce  traffic  jams  and  public 
inconvenience 

* Develop  early  maintenance  procedures,  such  as  non-destructive 
evaluation  (NDE)  capabilities  to  correct  flaws  before  they  get 
big 

* Plan  ahead 

* Improve  materials  and  emplacement  techniques 

* Improve  traffic  management  and  control  during  maintenance  and 
repair  (50%  of  the  repair  budget  is  spent  on  traffic  management 
during  repair) 

* Improve  data  flow  with  management  and  project  information 
systems 

* Integrate  traffic  control  with  all  jobs  that  need  to  be  done  at 
one  time  (i.e.,  take  a systems  approach) 

* Extend/ integrate  computer-aided  manufacturing  techniques  into 
road  construction,  maintenance,  and  operations 

* Make  systems  user  friendly  (a  major  concern) 

* Design  appropriate  standards  and  specifications  (tradeoffs 
between  innovation  and  open  systems;  have  an  innovation 
acceptance  process) 
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* Incorporate  GPS  into  robots  and  equipment  for  position 
determination  on  the  job  site  (determining  where  equipment  is 
located  on  a job  site  can  be  costly) 

* Examine  far-term  (10-20  years)  technology,  support  academic 
research  and  development 

* Examine  safety  issues  for  robots,  such  as  the  ability  of  humans 
and  robots  to  work  safely  in  the  same  workspace  (there's  little 
past  experience) , and  fail-safe  behavior  for  the  robots 

After  the  five  working  groups  discussed  the  various  requirements, 
opportunities,  and  issues,  they  returned  to  the  plenary  session 
and  briefed  the  Workshop  on  their  recommendations. 


Working  Group  1 

Recommendations  by  Working  Group  1 included: 

* Using  robotics  for  further  compaction  at  longitudinal  joints 
(to  seal  the  joints  and  increase  the  life  of  the  pavement) 

* Develop  an  alternative  to  excavation,  or  material  removal,  in 
traffic 

* Develop  computerized  asbuilt,  scheduling,  etc.,  to  have  an 
integrated  system,  a total  information  process 

* Develop  positional  control  of  equipment  and  tools 

* Establish  operational  demonstration  programs  for  new  technology 
and  share  risks  with  equipment  developers:  the  federal  government 
should  lend  equipment  to  the  contractor;  then  as  technology 
becomes  more  accepted,  the  government  should  lease  the  equipment 
to  the  contractor;  finally,  the  manufacturer  sells  the  equipment 
to  the  contractor 

* Emphasize  teleoperation  in  the  near-term,  but  some  level  of 
autonomy  is  needed  to  increase  efficiency 

* Avoid  very  specialized  equipment 

* Change  the  process,  rather  than  simply  automating  the  existing 
process 

* Establish  a long-range  R&D  program 

* Integrate  DoD  and  DoE  robotics/automation  technology  into 
current  DoT  R&D  process 
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* Use  foreign  technology  as  appropriate 

* Canvass  the  private  sector  for  other  technology  applications 


Comments : 

* There  are  already  mechanisms  in  place  for  industry  to  share 
risks  with  government,  such  as  the  Cooperative  Research  And 
Development  Agreement  (CRDA) . 

* Evolution  (instead  of  revolution)  of  new  technology  is  not  the 
only  path  - the  Xerox  machine  did  not  evolve  from  carbon  paper. 


Working  Group  2 

Working  Group  2 suggestions  include: 

* Involve  labor  in  technology  development  (for  example:  the 
bricklayer's  union  is  in  favor  of  technology  and  automation,  but 
they  want  to  be  involved  in  the  introduction  of  the  technology  on 
the  site) 

* Automate  the  worker  training  process 

* Assess  road  and  bridge  conditions  using  NDE  (non-destructive 
evaluation)  techniques,  preferably  in  real-time 

* Automate  the  detection  of  personnel,  equipment,  and  material, 
for  collision  avoidance 

* Automate  materials  management,  to  locate  materials  (which 
reduces  waste)  and  enable  night-time  (or  limited  visibility) 
operations 

* Automate  traffic  management  (replacing  the  flagman  and  ensuring 
traffic  stays  outside  the  worksite) 

* Justify  using  and  investing  in  a new  technology  with  value 
engineering  and  life-cycle  costing  (show  how  the  technology 
contributes  to  life-cycle  benefits,  not  just  the  initial  costs) 

* Introduce  mechanisms  in  the  contracting  process  to  promote 
innovative  technology  (low  bidder  criteria  discourages  new 
technology  because  the  company  cannot  recoup  the  investment  cost) 

* Develop  mechanisms  to  evaluate  innovations 

* Introduce  performance  incentives  to  justify  expenditures  on 
new,  innovative  technologies 
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* Encourage  industry /government /academia  partnering 


Comments : 

* The  mind  set  in  the  U.S.  needs  to  change.  The  U.S.  cannot 
simply  adopt  the  European  cooperation  model  because  we  have  a 
different  society.  The  form  of  the  suggested  partnering  needs  to 
be  determined. 

* The  highway  field  can  look  to  explorations  of  partnering  by  the 
building  construction  industry.  Also,  some  states  are  engaging 
in  partnering  in  road  construction. 


Working  Group  3 

Working  Group  3 focused  on  the  following  problems: 

* Low  cost  bid  gets  the  job 

- This  tends  to  shunt  aside  new  technology.  In  the  U.S.,  the 
contractors  are  too  small;  perhaps  small  companies  should  merge 
(or  perhaps  this  will  happen  as  natural  progress) . Perhaps 
smaller  companies  can  share  expensive  equipment  (perhaps  through 
leasing  arrangements) . 

- The  bidding  structure  should  be  renovated  to  favor  new 
technology. 

* Safety 

- Traffic  control  through  construction  sites  is  needed. 

Consider:  portable,  mobile,  automated  speed  bumps;  or  optimum 
detours  across  jurisdictional  boundaries  (to  reduce  driver 
frustration) ; or  defensive  radar  with  double  penalties. 

- Safety  is  labor  intensive;  new  technology  can  reduce  wasted 
time  and  labor. 

* Technology  transfer 

- The  technology  needs  CALTRANS  type  investment. 

- Incremental  introduction  of  new  technology  is  less  painful. 

- Educate  users  - otherwise  the  technology  won't  get  used. 

- Government  should  orchestrate  matched  partnerships 

* Regulation 

- Simplify  and  reduce  paperwork.  For  example,  perform  nuclear 
density  testing  at  roadside;  and  with  respect  to  asphalt  content, 
certify  plants,  use  process  reports,  and  eliminate  hazardous 
waste. 
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Comment : 

* The  problem  is  not  the  low  bid  system,  but  getting  the 
technology  into  the  bid.  Initial  costs  must  be  overcome  to  gain 
the  eventual  benefit  of  less  expensive  operational  or  other  costs 
(such  as  environmental  costs) . 


Working  Group  4 

Working  Group  4 considered  the  following  needs: 

* A database  of  automation  projects  in  progress  (perhaps 
generated  at  NIST) . And  get  useful  and  motivational  information 
to  contractors,  including  cost  and  savings  information.  "We  all 
want  information." 

* A listing  of  automation  demonstrations  - where  and  when. 

* Working  teams  between  contractor  and  administration,  to  ensure 
appropriate  technology. 

* Database  of  applicable  technologies  (such  as  ground-penetrating 
radar) . 

* Risk  liability  for  R&D  directed  projects.  (Who  has  the  most 
risk,  and  how  is  risk  shared)? 

* Low  bid  mentality  for  value  added  automation  projects. 

* Top  down  input  for  projects;  bottom  up  input  for  technology 
(contractors  know  what  the  problems  are  and  what  they  want) . 
Federal  government  should  recommend  projects.  States  should 
request  bids  with  new  technology  included.  Contractors  should 
determine  how  to  get  new  technology  into  proposal.  Contractors 
should  be  part  of  the  design  phase  of  new  technology. 

Working  Group  4 priorities  for  automation  and  robotics  included: 

* For  bridges 

- Remote  access 

- Remove  humans  from  danger 

- Miniaturized  equipment  to  operate  in  confined  space  and  toxic 
atmosphere 

- Continuous  on-line  monitoring  (load  cells,  acoustic  emission, 
weight  in  motion) 

- Use  of  improved  materials 


* For  traffic  safety 
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- Traffic  control  (determine  real  hazards  and  best  type  of 
traffic  control) 

- Information  about  road  repairs:  imaging  systems  to  find 
cracks,  etc;  preplanning  before  repairs  to  reduce  time  in  the 
field,  time  for  repairs,  and  public  inconvenience. 

- Educate  the  public  about  automation/robotics 

- Use  automation  for  backfilling  of  pipes 


Working  Group  5 

Working  Group  5 suggested: 

* Performance-based  specifications  that  would  allow  contractors 
to  use  proprietary  materials  and  technology. 

* Safety  developments  that  reduce  workplace  hazards  (whether 
contractor's  employees  or  traveling  public). 

* Partnerships  between  government,  academia,  and  industry,  to 
focus  on  use  of  existing  resources  and  technology.  Historically, 
new  technology  has  been  introduced  by  state's  mandating  certain 
technology,  or  performance  which  required  new  technology  to 
achieve. 


FINAL  PLENARY  SESSION 

After  the  presentations  of  the  working  groups,  various 
recommendations  culled  from  the  various  topics  and  issues  were 
listed.  Each  attendee  was  given  five  votes,  and  asked  to  vote 
once  for  each  of  his  or  her  top  five  priority  choices.  While  the 
list  consisted  of  a non-homogeneous , non-coherent , incomplete 
aggregation  of  issues  and  functional  areas,  it  is  a reasonable 
first-cut  effort  at  bounding  and  partitioning  the  problem  space. 
It  is  useful  to  see  what  a cross-section  of  the  road  construction 
community  thinks  is  important,  and  there  were  clear  demarcations 
of  the  topic  priorities.  The  list  follows,  in  priority  order 
with  the  number  of  votes  in  brackets  at  the  end  of  each  item. 

{1}  Use  lifecycle  value  engineering  or  performance-based 
specifications  to  provide  incentives  for  automation  and  robotics 
innovations  (such  as  designing  structures  or  bridges  for  ease  of 
later  maintenance  and  repair;  even  if  the  initial  cost  is 
greater,  the  lifecycle  cost  could  be  less) . [26] 

{2}  Use  automation  and  robotics  to  reduce  workplace  hazards.  [24] 

{3}  Develop  a site  positioning  system  (to  determine  what  is  on  a 
site  and  where  it  is) . [19] 
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{4}  Use  partnering  to  share  costs  and  risks  of  innovation 
(including  R&D) . [15] 

{ 5 } Develop  techniques  for  non  destructive  evaluation  (NDE)  and 
monitoring  for  bridges,  pavements,  locating  utilities,  etc.  [13] 

{6}  Improve  traffic  control  and  reduce  its  costs  (which  can  be 
50%  or  more  of  a repair  job)  to  highway  department  users.  [12] 

{7}  Assemble  a database  of  automation  projects,  demonstrations, 
and  technologies.  [12] 

{8}  Use  automation  and  robotics  for  pavement  maintenance  and 
construction.  [11] 

{9}  Use  automation  in  the  maintenance  of  bridges  (there  are 
500,000  bridges  in  the  U.S.  and  25%  are  known  to  be  structurally 
defective ) . [ 10 ] 

{10}  Design  automation  and  robotics  into  the  construction, 
maintenance,  and  retrofit  process  (i.e.,  concurrent  engineering); 
to  have  repair-friendly  designs  for  bridges,  for  example,  so  that 
a modified  bridge  structure  could  accommodate  new  technology 
using  sensors,  rails,  etc.  [9] 

{11}  Involve  the  labor  force  in  the  process  to  give  them  a stake 
in  innovation  (i.e.,  make  labor  a part  of  the  solution  instead  of 
part  of  the  problem).  [7] 

{12}  Develop  an  "as  built"  (and  "as  maintained"  and  "as 
operated")  integrated  project  information  system.  [7] 

{13}  Use  automation  and  robotics  to  enhance  the  capabilities 
(i.e.,  increase  efficiency)  and  reduce  the  cost  of  labor.  [7] 

{14}  Use  automation  and  robotics  for  trenching  and  ditching.  [3] 

{15}  Use  automation  and  robotics  for  demolition/recycling.  [0] 

While  one  might  partition  the  results  in  various  ways,  it  seems 
reasonable  to  assign  recommendations  {l}-{4}  to  the  first  rank  of 
priorities;  recommendations  {5}-{7}  (where  {7}  is  at  50%  of  the 
top  score  for  {1})  to  the  second  rank;  and  recommendations  {8}- 
{15}  to  the  third  rank. 

According  to  the  top  four  priorities,  the  automation  and  robotics 
program  should  take  the  broad  systems  approach  to  introduce  the 
technology,  stressing  lifecycle  value  engineering  for  highway 
projects.  While  increased  safety  should  be  a major  goal  for  the 
technology,  a key  initial  application  should  be  site  positioning 
system.  And  government,  industry,  and  academia  should  work 
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together  to  develop  and  introduce  the  technology  to  the  worksite. 


Consensus  For  An  Ongoing  Process 

After  the  plenary  voting  on  recommendations,  the  workshop 
consensus  was  that  there  should  be  an  ongoing  process  in  which 
government,  industry,  and  academia  could  work  together  to  further 
the  development  and  implementation  of  automation  and  robotics  for 
road  construction,  maintenance,  and  operations,  perhaps  a 
structure  analogous  to  what  IVHS  America  does  for  the  IVHS 
program.  Other  associations  could  be  involved  in  an  ongoing 
process,  such  as  the  Transportation  Research  Board,  the  Society 
of  Automotive  Engineers,  and  the  Stone,  Asphalt,  and  Concrete 
Pavement  Associations.  The  Association  For  Unmanned  Vehicle 
Systems  offered  to  sponsor  an  Industry  Support  Group  (ISG) , 
similar  to  the  ISGs  it  support  for  unmanned  air  and  ground 
vehicles.  Meanwhile,  NIST  will  provide  the  coordinating  function 
for  ongoing  working  groups,  and  it  will  survey  workshop  attendees 
and  other  prospective  participants  for  the  groups. 


Ernie  Kent:  Final  Words 

This  has  been  an  initial  meeting  in  a series  of  actions  - to 
touch  base  with  the  community  familiar  with  highway  construction, 
maintenance,  and  operations,  and  to  give  guidance  to  the  process. 
We  are  also  doing  a literature  search  , forming  panels  of 
robotics  experts  for  job  site  visits,  and  convening  a technology- 
based  workshop  in  conjunction  with  the  National  Science 
Foundation. 

Output  from  this  workshop  and  continuing  consultation  will  give 
us  an  ongoing  reality  check.  Out  of  the  process  will  come  a 
number  of  proposals  which  will  undergo  cost/benefit  analyses 
before  being  submitted  to  the  FHWA.  The  FHWA  will  then  act  on 
the  findings. 

Each  attendee  to  this  Workshop  will  get  a copy  of  a report  on  the 
Workshop.  We  encourage  you  to  act  as  a sounding  board  and  give 
us  suggestions  to  keep  us  on  track,  and  continue  to  provide  input 
into  the  process. 

We  have  had  a good  beginning  in  this  Workshop  and  some  of  our 
cherished  views  have  changed,  and  that  is  all  to  the  good.  We 
need  to  have  the  technologist's  views  changed  in  many  instances 
by  the  people  who  are  out  there  where  the  work  is  going  on.  This 
is  crucial  to  the  process. 

Thank  you  for  your  time,  your  effort,  and  your  input. 


WORKSHOP  REPORT 


24 


APPENDIX  A 


LIST  OF  WORKSHOP  ATTENDEES 
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LIST  OF  ATTENDEES 


Mark  G.  Alderman 
1221  E.  Broad  Street 
4th  Floor,  Maintenance 
Richmond,  VA  23219 
(804)  786-2801 

Jeff  Amoriello 
State  Highway  Administration 
9300  Kenilworth  Avenue 
Silver  Spring,  MD  20910 
(301)  220-7300 

George  L.  Bartholomew,  Jr. 

David  A.  Bramble,  Inc. 

P.O.  Box  419 
Chestertown,  MD  21620 
(410)  778-3023 

Tommy  Beatty 

Technology  Assessment  Branch  HTA-11 
400  7th  Street,  S.W. 

Washington,  DC  20590 
(202)  366-8028 

Frank  Bednar 

Technology  Assessment  Branch  HTA-11 
400  7th  Street,  S.W. 

Washington,  DC  20590 
(202)  366-8027 

Richard  L.  Bennett 

Prince  Georges  Contractors 

5411  Kirby  Road 

P.O.  Box  145 

Clinton,  MD  20735 

(301)  297-5230 

Leonhard  E.  Bernold 

Civil  Engineering  Department 

North  Carolina  State  University 

Box  7908,  Mann  Hall 

Raleigh,  NC  27695-7908 

(919)  515-3677 
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Donald  Boyd 

Pacific  Northwest  Lab. 

MSIN : K5-10 
P.0.  Be  999 
Richlan^,  WA  99352 
(?  9)  375-2149 

Caroline  Carver 

American  Traffic  Safety  Service  Association 
5440  Jefferson  Davis  Highway 
Fredericksburg,  VA  22405 
(703)  898-5400 

Greg  Chirikjian 

Dept,  of  Mechanical  Eng. 

Johns  Hopkins  University 
3400  North  Charles  Street 
Baltimore,  MD  21218 
(410)  516-7127 

Thomas  Chrisman 

Virginia  Department  of  Transportation 
P.O.  Box  2249 
Staunton,  VA  24402-2249 
(703)  332-9078 

Everette  L.  Cole 
1221  E.  Broad  Street 
4th  Floor,  Maintenance 
Richmond,  VA  23219 
(804)  786-2801 

Campbell  Crawford 

National  Asphalt  Pavement  Association 
5100  Forbes  Blvd. 

Lanham,  MD  20706-4413 
(301)  731-4748 

Richard  Daugherty 

Virginia  Road  and  Transport  Builders 
30  LaBrock  Drive 
Richmond,  VA  23255 
(804)  276-3393 

Jay  Etris 

Civil  Engineering  Research 
1015  15th  Street,  N.W. 

Suite  600 

Washington,  DC  20005 
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Robert  Finkelstein 
Robotic  Technology,  Inc. 
10001  Crestleigh  Lane 
Potomac,  MD  20854 
(301)  762-1622 

Kerien  Fitzpatrick 
Carnegie  Mellon  University 
Field  Robotics  Center 
Pittsburgh,  PA  15213 
(412)  268-6564 

Mark  Flack 

State  Highway  Administration 
9300  Kenilworth  Avenue 
Greenbelt,  MD  20770 
(301)  220-7300 

Charlie  Gildon 
Shirley  Contracting  Corp. 
8435  Backlick  Road 
Lorton,  VA  22079 
(703)  550-7897 

Ken  Goodwin 
NIST 

Bldg.  220,  Rm.  B124 
Gaithersburg,  MD  20899 
(301)  975-3421 

John  Gray 
Gray  & Associates 
7306  Radcliffe  Drive 
College  Park,  MD  20740 
(301)  474-7522 

John  L.  Gross 
NIST 

Bldg.  226,  Rm.  B158 
Gaithersburg,  MD  20899 
(301)  975-6068 

Robert  Hambright 
Southwest  Research  Institute 
6220  Culebra  Road 
San  Antonio,  Texas  78228 
(512)  522-2623 
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Martin  Herman 
NIST 

Bldg.  220,  Rm.  B124 
Gaithersburg,  MD  20899 
(301)  975-3452 

John  Huchrowski 

State  Highway  Administration 

707  North  Calvert  Street,  Room  403 

Baltimore,  MD  21202 

(301)  333-1122 

Von  Jennings 
Martin  Marietta 
103  Chesapeake  Park  Plaza 
Baltimore,  MD  21220 
(410)  682-0892 

Avi  Kak 

Robot  Vision  Lab 
School  of  Electrical  Eng. 

Purdue  University 

West  Lafayette,  IN  47907 

(317)  494-3551 

Thomas  Kelly 
CRAFCO 

10  Grandview  Drive 
Latham,  NY  12110 
(518)  783-8027 

Thomas  J.  Kelly 
Corps  of  Engineers 
2902  Newmark  Drive 
Champaign,  IL  61821-1076 
(217)  352-6511 

Ernest  Kent 
NIST 

Bldg.  220,  Rm.  B124 
Gaithersburg,  MD  20899 
(301)  975-3441 

Martin  Knecht 

Maryland  Department  of  Transportation 

SHA  Maintenance 

10  Elm  Road 

BWI  Airport,  MD  21240 

(301)  859-7311 
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Marshall  Klinefetter 
David  A.  Bramble,  Inc. 

P.O.  Box  245 

Wye  Mills,  MD  21679 

(410)  758-3676 

Robert  Latham 

Maryland  Highway  Contractors  Association 

Suite  707,  Empire  Towers 

7310  Ritchie  Highway 

Glen  Burnie,  MD  21061 

(410)  760-9505 

Ronald  Lumia 
NIST 

Bldg.  220,  Rm.  B127 
Gaithersburg,  MD  20899 
(301)  975-3452 

Bernard  McCarthy 

The  Asphalt  Institute 

6917  Arlington  Rd. , Suite  210 

Bethesda , MD  20814 

(301)  656-5824 

Chuck  McGogney 

Physical  Research  Division 

FHWA-TFHRC  (HAR-20) 

6300  Georgetown  Pike 
McLean,  VA  22101-2296 
(703)  285-2441 

Donald  Merwin 

Highway  & Heavy  Construction  Products  and  Infrastructure 

Cahners  Publishing 

1350  East  Touhy  Avenue 

P.O.  Box  5080 

Des  Plaines,  IL  60017-5080 

(708)  390-2627 

Larry  Michael 

Maryland  State  Highway  Administration 
2323  West  Joppa  Road 
Brooklandville , MD  21022 
(410)  321-3538 

Michael  Moravec 
Federal  Highway  Administration 
Leo  W.  O'Brien  Building  Rm.  729 
Albany,  New  York  12207 
(518)  472-4253 
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Salim  Nassif 

Technology  Assessment  Branch  HTA-11 
400  7th  Street,  S.W. 

Washington,  DC  20590 
(202)  366-8026 

Charles  Niessner 
Federal  Highway  Administration 
6300  Georgetown  Pike 
Mail  Stop  HAR-20 
McLean,  VA  22101-2296 
(703)  285-2074 

Arnold  O'Donnell 
O'Donnell  Construction  Co. 

2209  Channing  Street,  N.E. 
Washington,  DC  20018 
(202)  259-3350 

Jay  Openshaw 

Cherry  Hill  Construction,  Inc. 

8211  Washington  Blvd. 

P.O.  Box  356 
Jessup,  MD  20794-0356 
(410)  799-3577 

Morton  Oskard 

Federal  Highway  Administration 
6300  Georgetown  Pike 
Mail  Stop  HAR-10 
McLean,  VA  22101-2296 
(703)  285-2074 

Thomas  J.  Pasko,  Jr. 

Department  of  Transportation 
HAR-1 

6300  Georgetown  Pike 
McLean,  Virginia  22101-2296 
(703)  285-2034 

Richard  Quintero 
NIST 

Bldg.  220,  Rm.  B127 
Gaithersburg,  MD  20899 
(301)  975-3456 

Arthur  Sanderson 

Rensselaer  Polytechnic  Institute 
Troy,  NY  12180 
(518)  276-2879 
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Leonard  Schultz 

Maryland  Department  of  Transportation 

SHA  Maintenance 

10  Elm  Road 

BWI  Airport,  MD  21240 

(301)  859-7311 

Robert  Sewell 

Genstar  Stone  Products  Co. 

11350  McCormick  Road 
Hunt  Valley,  MD  21031 
(410)  527-4000 

Richard  Stander 

Mansfield  Asphalt  Paving  Co. 

P.O.  Box  1321 
Mansfield,  Ohio  44901 
(419)  529-8455 

Richard  Steele 
1401  E.  Broad  Street 
Materials  Division 
Richmond,  VA  23219 
(804)  786-2801 

Yair  Tene 

Global  Associates,  Ltd. 

Suite  205 

2300  Clarendon  Blvd. 

Arlington,  VA  22201 
(703)  351-5660 

Karen  Trovato 
Phillips  Lab. 

345  Scarborough  Road 
Briarcliff  Manor,  NY  10510 
(914)  945-6233 

David  Urbane 
Caterpillar,  Inc. 

100  NE  Adams  Street 
Peoria,  Illinois  61629 
(309)  675-4301 

Douglas  Walters 
McLean  Contracting  Co. 

6700  McLean  Way 
Glen  Burnie,  MD  21060 
(410)  553-6700 
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James  Wentworth 

Federal  Highway  Administration 

6300  Georgetown  Pike 

Mail  Stop  HAR-10 

McLean,  VA  22101-2296 

(703)  285-2748 

Thomas  West 

California  Dept,  of  Transportation 
Division  of  New  Technology 
5900  Folsom  Blvd. 

Sacremento,  CA  95819 
(916)  454-6583 

Gary  White 

Federal  Highway  Administration 
18209  Dixie  Highway 
Homewood,  Illinois  60430 
(708)  206-3221 

Meg  Willett 

American  Road  & Trans.  Builders  Association 
501  School  Street,  S.W. 

Washington,  DC  20024-2713 
(202)  488-2722 

Charles  Woo 

Federal  Highway  Administration 
6300  Georgetown  Pike 
Mail  Stop  HAR-20 
McLean,  VA  22101-2296 
(703)  285-2444 

Richard  Wright 
NIST 

Bldg.  222,  Rm.  B216 
Gaithersburg,  MD  20899 
(301)  975-5900 
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APPENDIX  B 


LITERATURE  AVAILABLE  AT  NIST 
RELEVANT  TO  AUTOMAT I ON /ROBOT ICS 
FOR  ROAD  CONSTRUCTION,  MAINTENANCE,  AND  OPERATIONS 
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Abraham,  Dulcy  M.  and  Leonhard  E.  Bernold,  "A  State-Based 
Framework  For  Construction  Control,"  Computing  in  Civil 
Engineering  And  Symposium  On  Data  Bases,  Proceedings  of  the 
Seventh  Conference,  Washington,  DC,  American  Society  of  Civil 
Engineers,  May  6-8,  1991 

Abraham,  Dulcy  M.  and  Leonhard  E.  Berhold,  "Control  For  Computer- 
Integrated  Construction,"  Computing  in  Civil  Engineering: 
Computers  in  Engineering  Practice,  Proceedings  of  the  Sixth 
Conference,  Atlanta,  GA,  American  Society  of  Civil  Engineers, 
Sept.  11-13,  1989 

Advanced  Highway  Maintenance  Technology  Program,  (Data  Sheets, 
including:  Delineation  and  Signing  Technology;  Hazmat  Debris  and 
Litter  Removal;  Landscape  Management;  Workzone  Safety;  Pavement 
Integrity;  Structures  Maintenance;  Maintenance  Minded 
Infrastructure;  Paint  Striping  Guidance  System;  Paint  Striping 
Information  System;  Maintenance  of  the  Future;  Remote  Hazmat  Lab; 
Smart  Herbicide  Applicator;  Litter  Bag  Retrieval;  Telerobotics  in 
CALTRANS  Maintenance;  Automated  Raised  Marker  Placement; 
Crack/Joint  Sealing  Machine) , California  Dept,  of  Transportation, 
Sacramento,  CA,  1992 

Albus,  James,  et.  al.,  "Mining  Automation  Real-Time  Control 
System  Architecture  Standard  Reference  Model  (MASREM) : Coal  Mine 
Automation,"  NIST  Technical  Note  1261,  Volume  1,  1989 

Albus,  James  and  Ken  Goodwin,  "The  NIST  Spider:  A Robot  Crane," 
National  Institute  of  Standards  and  Technology,  Gaithersburg,  MD, 
1992 

Ambrose,  Dean,  "Using  Speech  Technology  in  the  Mining  Industry," 
Speech  Technology,  Oct/Nov  1989,  pp.  34-37 
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high  potential  for  future  application. 


i 


m 

4S 

r~ 

4J 

<M  4-> 

U 

0-h  n 

* 

0) 

£ 4J 

T> 

4J  O 

* 

«* 

0 

CSS 

r> 

r> 

k 

• #T-i 

in 

a 

0 «P  © 
k B k 

+> 

• ©>  a 

c 

a to 

m 

E 

a 

o 

H 

© 

> 

© 

Q 

•a 

c 

m 

JZ 

O 

u 

0 

0) 

(0 

• 

K 

•• 

in 

CM 


<0 


0 

© 


m 

4J 

10 

a 


a 

>*  m 

* C 

© m o 

C t5  B-h 
3 c m 4J 

0 4J  0 

Cfi  I4  «0 

C 4J  >1  0)  fH 

O'  c w a c © 

•H  O'  © O © *H 

(0  C E C E 
0)  *H  0)  o O 0) 

Q C O'-H  -H  0**0 

C B +>  Vi  0 c 
*0  0 G 0 <M  C 0 

• h © i « © 

n a x u u x >i 

•H  Oh  k 
k *©  • %4  O 

© C (A  C >1  C 4J 

B © H B B B 

2 5 * k 

a b js  o 

B «P  O'  .Q 
O B «H  B 

o Q x ^ 


Collection  and  Analysis 
Construction  Management  and  15% 

Quality  Control 

Highway  Information  Systems  il% 

and  User  Communications 


Table  2.6:  'Use  of  Computer  Technology  in  State  DOTs  - 1985 
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! CREATE  "PULL"  FOR  TECHNOLOGY 

'T he  U.S.  Gov't:  - 


Owns: 

230,000  MILES  OF  HWY. 
(6%) 

Provides  Fed 

Aid  to: 

600,000  MILES  OF  HWY. 
(16%) 

Supports: 

270,000  Bridges  on 

Fed  Aid  (47%)  • 

Maintains: 

2800  MILES  OF  * 

Railroad 

Owns: 

417,000  Buildings 

Leases: 

68,000  Locations 

Administers: 

622M  ACRES  OF  LAND 

(30%) 

Maintains: 

300M  So.  Ft.  of 

Low  Slope  Roofs  (Army) 

Builds 

♦ 

Prisons,  Post  Offices, 
Military  Installations 
and  Airfields, 

Veterans  Hospitals, 
Coastal/Hydraulic 
Installations,  etc. 

SECTION:  10 
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Why  the  Workshop? 

The  purpose  of  the  workshop  was  to  attract 
selected  researchers  and  practitioners  from  academia, 
government  and  industry  to  exchange  information, 
ideas  and  their  vision  on  how  "Automated 
Construction  and  Excavation"  can  benefit  Civil 
Infrastructure  Systems  (CIS). 

The  size  and  quality  of  the  world  competition  in 
the  area  of  surface  and  underground  construction 
and  excavation  technologies  dictated  an  immediate 
plan  of  action.  The  workshop  offered  the  opportunity 
to  professionals  of  different  background  to  interact 
with  each  other,  be  exposed  to  different  philosophies, 
and  contribute  in  identifying  a coherent  set  of 
recommendations  for  research  that  will  produce  the 
highest  returns. 


Objectives  of  the  Workshop 

The  general  theme  of  the  meeting  was 
"Automation  in  Construction  and  Excavation 
Technology"  with  the  following  objectives: 

♦ Present  an  inventory  of  state-of-the-art 
procedures  in  construction  and  excavation 
technologies 

♦ Identify  application  areas  where  these 
technologies  will  have  an  immediate  return 
(Transportation,  Environmental  Protection, 
Utility  Networks,  and  others) 

♦ Produce  a set  of  recommendations  for  research 
needs  and  identify  potential  near  and  long-term 
programs. 


Organization  of  the  Workshop 

The  NSF,  NIST  and  FHWA  sponsors  felt  that  this 
is  the  time  to  obtain  a holistic  view  of  the  challenges 
facing  the  construction  and  excavation  industry.  In 
today's  stringent  safety  and  performance  requirements 
there  is  a need  to  address  the  construction  and 
excavation  problem  from  a global  perspective  The 
common  theme  bonding  all  the  contributors  to  this 
workshop  was  "Automation  and  Machine  Intelligence 
in  Surface  and  Underground  Construction." 

The  broad  range  of  automated  construction 
/excavation  machines  includes: 

♦ the  automated  earth-moving,  spreading, 
compacting  machines,  and  Whittaker's  family  of 
autonomous  machines  at  one  end  of  the 
spectrum 

♦ the  continuous  mining  machines  in  coal  mines, 

♦ the  continuous  Tunnel  Boring  Machines  in  hard 
rock  and  weak  soils, 

♦ the  drill  and  blast  machines,  and 

♦ microtunneling  machines  at  the  other  end  of 
spectrum. 

All  of  these  machines  are  "semiautonomous"  with 
shared  man-machine  control,  operating  in  a highly 
unstructured  environment.  Automation  is  the  common 
link  between  all  these  machines,  which  although 
different  in  size  and  function  encompass: 

1.  The  automatic  movement  (guided  motion)  of 
the  machine 

2.  The  automatic  manipulation  of  appropriate 
tools  for  the  realization  of  predetermined  tasks 

3.  The  automatic  sensing  and  processing  of 
real-time  data  for  decision  making  and  control 
at  the  local  scale 

4.  The  automatic  characterization  (detection)  of 
the  operating  environment  at  the  global  scale 
(macroscale). 
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The  selected  topics  for  discussion  covered,  in 
twelve  technical  sessions,  a broad  spectrum  of 
application  areas,  from  the  highly  automated 
excavation  devices  at  the  surface  of  the  soil  medium, 
to  the  sophisticated  TBM's  (Tunnel  Boring 
Machines),  the  continuous  mining  machines  used  in 
the  coal  mining  industry  and  the  different  small 
diameter  boring  machines  used  in  trenchless 
technology. 


Content  of  Technical  Sessions 


The  technical  areas  covered  in  each  session  are 
illustrated  below: 


In  Sessions  1 to  4 the  focus  was  on  areas  of 
automation  as  applied  to  highway  construction  and 
surface  operations,  while  in  sessions  5 to  11  the 
emphasis  was  on  underground  excavation  and 
operations  related  to  the  development  of  the 
underground  space.  The  topics  discussed  in  each 
session  are  described  below: 

Session  1 (Chaired  by  Arthur  Sanderson)  focused  on  the 
design  for  automation  in  highway  construction,  site  integration 
through  hierarchical  control,  and,  automated  project  planning 
and  scheduling.  Road  construction  and  maintenance  require 


extensive  coordination  of  workers,  machines,  and  resources. 
The  use  of  advanced  computer  and  automation  technologies 
provide  the  means  to  improve  the  efficiency,  productivity,  and 
safety  of  construction  projects. 

Session  2 (Chaired  by  Kcricn  Fitzpatrick)  placed  the 
emphasis  on  technologies  for  automated  earth-moving, 
spreading,  compacting,  lifting  and  positioning  of  materials 
and  structural  elements.  Computers  and  communications 
technology  have  revolutionized  earth-moving  industries.  The 
major  changes  are  still  to  come,  but  they  are  just  around  the 
comer.  These  new  developments  include  basic  communication, 
machine  monitoring  and  diagnostics,  job  and  business 
management,  planning  and  operations. 

In  Session  3 (Chaired  by  Leonhard  Bemold) 
presentations  were  made  on  teleoperated  devices,  smart  tools, 
operator-assisted  automation,  advanced  operator  interfaces,  and 
virtual  reality.  The  creation  of  intelligent  controls  for  large 
and  heavy  machines  used  in  the  construction  of  highways  poses 
a considerable  challenge  to  engineers  and  scientists.  For 
example,  the  unstructured  nature  of  soil  in  its  natural  setting 
requires  a thorough  understanding  of  soil  mechanics  to  develop 
dynamic  control  systems  for  robotic  excavation.  Problems  that 
originate  from  site  conditions  in  which  construction  operations 
have  to  take  place  need  urgently  to  be  solved. 

Session  4 (Chaired  by  Avi  Kak)  dealt  with  technologies 
for  inspection  of  bridges  and  road  surfaces,  automated 
surveying,  "As-Built"  databases,  site  positioning  and  quality 
assurance.  The  focuses  of  ongoing  research  activities  are  on 
nondestructive  testing  of  highway  and  runway  pavements,  and 
the  application  of  this  technology  to  real  time  sensing  of  the 
quality  control  of  repairs  or  new  construction. 

Session  5 (Chaired  by  William  Whittaker)  discussed 
elements  of  automated  excavation,  hazardous  waste 
applications,  military  applications,  academic  and  corporate 
research.  Advances  in  perception,  reasoning  and  manipulation 
that  have  it  technically  feasible  for  a robot  to  discern  objects, 
discriminate  them  from  their  surroundings,  plan  approach 
trajectories  and  grasp  them.  However,  an  important  class  of 
material  handling  problems  related  to  the  extrication  of  objects 
that  are  embedded  in  soils  need  to  be  solved. 


In  Session  6 (Chaired  by  Priscilla  Nelson)  new  TBM 
technologies  were  presented,  along  with  technologies  for 
steering  and  control  systems,  perception  sensors,  automatic 
lining.  New  developments  in  TBM's  design  include:  main  beam 
steering,  floating  grippers,  direct  drive  cutterheads,  mechanical 
cutterhead  stabilization,  hydraulic  clutch  engagement,  oil 
sealing  system  for  cutterhead  drive,  effective  ventilation  and 
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dust  control,  field  replaceable  cutter  assemblies,  new  safety 
features. 

Session  7 (Chaired  by  Ken  Stokoe)  focused  on 
geophysical  methods  for  subsurface  detection,  site 
characterization  and,  subsurface  utility  engineering.  New 
developments  in  fusing  certified  three-dimensional  data  of  soil 
conditions  and  underground  existing  utilities  into  the  field 
operator  console  of  "smart"  equipment.  This  would  guide 
automated  directional  excavating  machines  to  avoid  existing 
underground  structures.  However,  presently  geophysical 
exploration  is  still  relying  on  old  technologies  that  do  not  use 
the  recent  developments  in  automation. 

Session  8 (Chaired  by  Ray  Sterling)  elaborated  on 
today’s  problems  and  opportunities  in  R&D  for  excavation  by 
blasting.  Conventional  drill  and  blast,  while  being  able  to 
excavate  the  hardest  of  rocks  at  acceptable  efficiencies,  are 
limited  in  that  the  technique  must  be  applied  in  cyclic  fashion, 
resulting  in  the  inefficient  and  often  interfering  use  of  the 
equipment  required  for  each  cycle.  Newly  developed 
technologies  are  shown  to  be  energy  efficient  for  breaking  hard 
rocks  and  permit  more  continuous  and  automated  operation. 

Session  9 (Chaired  by  Tom  Iscley)  dealt  with  the 
tracing  and  steering  of  horizontal  earth  boring  systems,  recent 
microtunneling  innovations  and  applications  of  trenchless 
technology.  Trenchless  technolog}  is  the  process  of  installing 
or  rehabilitating  underground  infrastructure  with  minimum 
disruption  and  destruction  typically  associated  with  traditional 
methods.  There  are  many  methods  that  make  up  the  family  of 
techniques  that  can  be  used  to  install  new  infrastructure 
system 

Session  10  (Chaired  by  Basile  Dcndrou)  provided  the 
framework  in  which  most  of  the  new  technologies  introduced  in 
the  previous  sections  was  put  together  in  an  integrated  computer 
based  environment  to  support  the  implementation  of  Mega 
excavation  projects.  These  technologies  included:  an  automated 
engineering  information  system,  a reactive  navigation  scheme, 
real-time  position  measurement  in  underground  construction, 
robotic  perception  of  material  properties,  dynamic  interface 
simulation  for  underground  construction  operations.  It  is 
believed  that  the  most  efficient  way  to  handle  the  mega-scale 
problem  of  underground  excavation  projects  is  with  the  use  of 
integrated  computer  platform  that  will  assist  in  the  management 
and  control  of  automation  as  applied  to  the  excavation  process. 
The  new  integrated  systems  expand  on  the  GIS  technology  to 
include  the  3rd  (depth)  dimension,  time,  and  the  interaction  of 
all  processes  characterizing  the  underground  excavation. 


Session  11  (Chaired  by  Herbert  Einstein)  continued  the 
general  theme  of  the  previous  session,  with  more  details  on  the 
information  technology  as  applied  to  construction,  mechanistic 
simulations  for  safety  analyses,  data  fusion  and  visual  data 
bases,  intelligent  information  systems.  Information  technolog} 
is  the  natural  link  between  different  activities  of  tunneling 
construction  that  includes  management-costing  programs,  and 
safety  and  risk  analyses. 

Finally  in  Session  12  (Chaired  by  Mike  Gaus)  the  impact 
of  the  new  excavation  technologies  on  the  construction  industry 
was  presented  through  different  evolutionary  and  visionary 
implementations.  Two  new  ideas  were  promoted  in  this  session, 
the  concept  of  underground  freight  network  and  the  concept  of 
underground  urban  corridors. 


Professionals  Attending  the 
Workshop 

With  the  large  variety  of  topics  covered  in  each 
session  it  was  only  natural  to  have  a broad  range  of 
professionals  of  different  background  participating  in 
this  workshop.  It  was  very  interesting  to  see: 
mechanical  engineers  interacting  with  civil  engineers, 
specialists  in  robotics  talking  to  specialists  in 
geomechanics,  construction  specialists  discussing  with 
manufacturers,  and  engineers  from  the  military  sharing 
their  experience  with  the  private  sector.  A glance  at 
these  new  concepts  and  ideas  resulting  from  these 
discussions  is  given  in  the  following  sections. 


Development  Trends  Identified  at 
the  Workshop 

Progress  in  surface  and  underground 
"Automation"  technology  is  necessary  if  the 
construction,  mining  and  environmental  protection 
industries  are  to  remain  competitive  in  the  world 
market.  The  workshop,  clearly  demonstrated  that 
recent  advances  in  automation  technologies, 
particularly  the  development  of  fast,  inexpensive 
computers,  control  software  design,  and  sensing 
technology,  improve  construction  efficiency  and 
worker  health  and  safety. 
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In  the  construction  site  of  the  very  near  future, 
construction  workers  and  operators  will  be  relocated 
from  the  relative  dangerous  construction  site,  to  a 
protected  Control  Center  from  which  the  operators 
will  be  able  to  direct  the  activities  of  their  machines 
through  graphic/video  real-time  computer  terminals. 
The  new  technologies  for  computer-assisted 
construction  are  being  developed  by  building  upon 
conventional,  mechanized  equipment  used  in  well- 
understood  construction  operations.  By  using  familiar 
machines  in  familiar  ways,  the  manufacturers  hope  to 
avoid  confronting  the  barriers  traditionally  associated 
with  introducing  radically  new  machines  and 
procedures.  This  trend  was  clearly  identified  in  many 
sessions  of  the  workshop  dealing  with  different 
excavation  machines.  Table  1 shows  typical  machines 
and  their  "automation"  components,  as  identified  at 
the  workshop. 


Benefits  of  the  Automated 
Construction  and  Excavation 


The  potential  benefits  of  "Automation"  in 
excavation  technologies  include  quicker  and  higher 
quality  site  characterization,  improved  craftspeople 
performance,  improved  modular  construction,  reduced 
rework,  improved  performance  and  quality 
improvement,  and  improved  overall  construction  time. 

The  benefits  according  to  preliminary  estimates 
given  by  Dr.  Yvan  Beliveau  can  be  more  than  $150 
billion  per  year  in  savings  from  the  $500  billion 
U.S.  Construction  industries.  Most  of  these 
spendings  are  planned  for  construction  at  the  surface. 
However,  the  underground  space  may  well  be  the  new 
frontier  for  the  U.S.  construction  industry. 

The  underground  excavation  industry  offers  a 
unique  opportunity  to  attract  the  interests  of  many 
different  professions  and  put  into  practice  new 
technological  concepts  and  ideas,  as  shown  in  the 


closing  session  of  the  workshop  Potential  new  surface 
and  underground  developments  for  the  next  5 years  in 
the  U.S.  alone  include: 

In  Surface  Transportation: 

♦ $5-$10  billion  dollars  for  the  rehabilitation  of 
highways 

♦ $20  billion  dollars  for  expanding  the  highway 
network 

Candidates  for  the  implementation  of  these  projects 
are:  Surface  excavators. 

In  Underground  Transportation: 

♦ $6-8  billion  dollars  for  new  programs  in  urban 
railway  transportation 

♦ $2  billion  dollars  for  new  highway  tunnels 

♦ $10  billion  dollars  during  first  5 years  and 
increasingly  thereafter  for  the  new  underground 
freight  systems 

Candidates  for  the  implementation  of  these  projects 
are.  TBM's,  Drilling  Machines. 

In  Mining: 

♦ $2  billion  dollars  for  new  Mines 

♦ $20  billion  per  year  for  ongoing  operations 

Candidates  for  the  implementation  of  these  projects 
are:  Mining  Machines,  Drilling  Machines,  Surface 
Excavators. 

In  Water  Management: 

♦ $5  billion  dollars  for  rehabilitating  existing  pipe 
networks 

♦ $6  billion  dollars  for  new  construction  of  water 
facilities 

♦ $4  billion  dollars  for  new  sewerage  networks 

Candidates  for  the  implementation  of  these  projects 
are:  TBM's,  Trenchless  technology,  and  surface 
excavators. 
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Table  1 Typical  Automated  Machines 


Machine  Type 

Features  and  Functions 

Surface  Excavators 

Semi-autonomous  machines  operator  assisted. 

Real-time  Positioning:  GPS,  (2*  accuracy) 

Automotive:  compt.  enhanced  automotive  fcts. 

Electronic  Hardware:  RISC  VLAN's  technology 

Software:  Assembly  and  C on  RO  chips. 

US  against  Foreign  Competition:  Ahead 

Whittaker's  Family  of  Machines 
(Primarely  for  material  handling  in  the  field) 

Remote-supervised  operating  system  (semi-autonomous) 
and  automous  machines 

Real-time  Positioning:  LPS  (0.2*  accuracy) 

Automotive:  Traditional  (Battery) 

Electronic  Hardware:  RISC 

Software:  Assembly 

US  against  Foreign  Competition:  Ahead 

Mining  Machines 

Computer-assisted,  remote-supervised  operating  system 
(semi -autonomous) 

teal-time  Positioning:  LPS  (Laser  Based) 

Automotive:  Electric  Power 

Electronic  Hardware:  RISC  • CISC 

Software:  Assembly  and  C,  Video  Console 

US  against  Foreign  Competition:  Ahead 

TBM  (Tunnel  Boring  Machine) 

Computer-assisted,  manually  remote-supervised 
operating  system  (semi-autonomous) 
leal-time  Positioning:  LPS  (Laser  Based) 

Automotive:  Traditional  Gas  /Electric 

Software:  Conventional,  Video  Console 

US  against  Foreign  Competition:  Weak 

Drilling  and  bolting  machines 

Automated  mechanically-assisted  operation, 
(semi-autonomous) 

leal-time  Positioning:  LPS  (Laser  Based) 

Automotive:  Traditional  Gas 

Electronic  Hardware  Traditional 

Software:  Assmbly  and  C 

JS  against  Foreign  Competition:  Even 

Trenchless  Technology 

Remote-supervised  operating  system  (semi-autonomous) 
Real-time  Positioning:  LPS  (Laser  Based) 

Automotive:  Traditional 

Electronic  Hardware  State  of  the  Art  RISC 

Software:  Assembly  and  C,  Video  Console 

US  against  Foreign  Competition:  Weak 
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In  Clean  up  Operations:  (Information  provided  by 
Dr.  Vernon  Myers) 

♦ $50  billion  for  2000  Superfund  sites 

♦ $100  billion  for  3,750  RCRA  sites 

♦ $ 1 0-$  1 5 billion  for  6,000  sites  of  DOD 

♦ $53-90  billion  for  45  sites  of  DOE 

♦ $100  billion  for  30,000  Real  Estate  sites 

Sixty  percent  of  these  cleanup  operations  require  the 
use  of  excavation  technologies.  Candidates  for  the 
implementation  of  these  projects  are:  TBM's, 

Trenchless  technology,  and  surface  excavators. 

Underground  utilities: 

♦ $0.5-1  billion  dollars  for  communication 
networks 

♦ $1  billion  dollars  to  create  underground  space 
in  major  cities 

Candidates  for  the  implementation  of  these  projects 
are:  TBM's,  Trenchless  technology  The  1992  World 
Market  for  trenchless  pipelaying  is  estimated  at 
approximately  $4.7  Billion  (Kramer,  1993) 

These  are  only  conservative  estimates  but  the 
important  fact  is  that  the  magnitude  of  these  new 
financial  ventures  is  such  that  "Automated  excavation" 
can  make  a significant  contribution  towards  the 
infrastructure  investments  that  are  necessary  to  help 
reverse  the  recent  downward  trend  of  the  U S. 
economy. 


Promoting  and  Expanding 
’’Automated  Excavation 
Technology" 

Technology  alone  is  not  enough  to  commit  the 
governmental  agencies  and  the  investors  to  these  new 
developments.  The  discussions  at  the  workshop 
clearly  indicated  that  there  is  a need  to  attract  the 
popular  concern  as  represented  through  the  legislative 
institutions  and  regulations  According  to  attorney 
David  Calverley,  environmental  regulations  in  the  area 


of  the  utilization  of  underground  space,  are  not  nearly 
as  well  developed  as  in  other  media  An  integrated 
regulatory  scheme  could  significantly  assist  in  the 
overall  growth  of  the  industry,  while  assuring  at  the 
same  time,  that  the  goals  of  sustainable  development 
are  met. 

However,  the  single  biggest  issue  to  the  overall 
success  of  these  technologies  is  to  properly  educate, 
retrain,  attract,  and  retain  well-qualified  professionals 
of  the  technology.  It  was  the  consensus  of  all  the 
participants  that  the  level  of  resources  that  the 
construction  industry  spends  for  training  and 
education  is  inadequate  (1%  of  its  sales  compared  to 
10%  of  sales  for  manufacturing).  To  remedy  this,  the 
participants  of  the  workshop  agreed  to  follow  the  Dr 
Herbert  Einstein's  advice  and  create  a "virtual 
network"  of  dedicated  professionals  that  will  actively 
support  the  promotion  of  "Automated  Excavation 
Technology." 

There  are  undoubtedly  costs  associated  with  the 
implementation  of  these  technologies.  In  the  final 
analysis,  however,  it  is  expected  that  each  dollar  of 
added  efTort  will  yield  tenfold  and  even  larger 
returns. 


Technical  Challenges 

The  new  developments  in  automated  surface  and 
underground  excavation  include: 

1/  the  identification  of  in  situ  conditions  using  the 
latest  technology  in  sensor  devices  (geophysical  site 
investigation),  2/  Processing  the  in  situ  information 
through  computers  and  expert  modules  to  establish 
design  and  construction  strategies,  3/  Adapt 
automatically  the  excavation  tools  and  proceed  with 
the  excavation,  4/  Remove  and  automatically  process 
the  excavated  material,  5/  Install  automatically  the 
lining  or  other  structural  system,  and  6/  Complete  the 
job  to  satisfy  building  code's  safety  requirements. 

To  implement  all  these  tasks  a broad  range  of 
different  technologies  must  be  blended  together  in  a 
macroengineering  framework  (macro-scale  approach). 
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The  challenge  now  is  to  integrate  these  technologies  in 
the  "semiautonomous"  excavation  machines,  operating 
in  the  highly  unstructured  environment  of  the  real 
world.  The  following  table  provides  a summary  of  the 
different  disciplines  required  for  the  "Automation"  of 
most  of  these  excavation  machines. 


Automated 

Function 

Disciplines  & Technologies 

Automatic  movement 

(guided  motion)  of  the 
machine. 

Robotics:  Robotic  control  and 
mobility.  Task  planning.  Intelligent 
sensors  and  actuators,  Automatic 
drilling  and  lining  systems. 
Manufacturing  Automation: 

Flexible  manufacturing,  Process 
automation,  Computer  integrated 
manufacturing. 

Automatic  manipulation 
of  appropriate  tools  for 
the  realization  of 

predetermined  tasks 

Control  Applications:  Motion 
control.  Guiding  systems, 

Modeling  and  simulation,  Signal 
processing.  Fuzzy  control  and 
diagnosis. 

Automatic  sensing  and 
processing  of  real-time 
data  for  decision  making 
and  control  at  the  local 
scale 

AI  & Expert  Systems:  Knowledge 
bank.  Intelligent  control,  AI 
software.  Network  dynamics, 
learning  algorithms,  hardware 
implementation 

Global  and  Local  Positioning: 
Laser  network. 

Infrared  technologies. 

Computer  Vision:  Image 

processing.  Dynamic  scene 

analysis,  Machine  vision.  Pattern 
recognition.  Fractals  and  IFS 
algorithms. 

Automatic 

characterization  and 

detection  of  the  operating 
environment  at  the  global 
scale  (macroscale). 

Automatic  Stabilizing 
counter  measures. 

Site  Characterization  and 
Detection:  Geophysical  methods, 
Real  time  sensors  of  evolution  of 
mechanistic  processes. 

Mechanistic  and  Construction 
Simulation:  Prototyping,  Parallel 
processing.  Impact  of  automation 
to  the  environment,  Reliability  and 
risk  analysis.  Management  and 
cost 

Soil/Rock  Stabilization:  Concrete 
admixtures,  Geotextiles,  Fiber 
anchors.  Chemicals  for  soil 
grouting  and  stabilization. 

The  overall  research,  development,  demonstration 
program  should  be  on  a 5 year  schedule  requiring  a 
total  funding  of  $15  to  20  million  dollars  for  a target 
machine  system.  TBM's  and  mining  machines  will 
require  more  research  funds  than  microtunneling  and 
material  handling  machines. 


Automated 

Function 

R & D Cost  Estimate 

Automatic  movement 

(guided  motion)  of  the 
machine. 

Robotics:  $4  million 

Manufacturing  Automation: 

$2  million 

Automatic  manipulation 
of  appropriate  tools  for 
the  realization  of 

predetermined  tasks 

Control  Applications:  $4  million 

Automatic  sensing  and 
processing  of  real-time 
data  for  decision  making 
and  control  at  the  local 
scale 

AI  & Expert  Systems:  $1  million 

Global  and  Local  Positioning: 

$1  million 

Computer  Vision:  $1  million 

Automatic 

characterization  and 

detection  of  the  operating 
environment  at  the  global 
scale  (macroscale). 

Automatic  Stabilizing 
counter  measures. 

Site  Characterization  and 
Detection:  $ 4 million 
Mechanistic  and  Construction 
Simulation:  $1  million 

Soil/Rock  Sciences:  $ 2 million 

These  key  technologies  can  be  tested  in  certain 
important  critical  missions,  such  as: 

♦ The  cleanup  operation  of  the  nuclear  power 
plants. 

♦ Other  major  government  excavation  projects. 

National  Geotechnical  Experimentation  Sites  (NGES) 
sponsored  by  NSF  and  FHWA  would  provide  the 
means  of  field-testing  these  technologies  in  the 
development  stages. 
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Recommendations 

Here  are  some  key  suggestions  for  what  the  research 
community  and  funding  agencies,  might  realistically  do 
to  foster  more  effective  research  in  the  area  of 
"Automated  Excavation  and  Material  Handling  in  the 
Field": 

♦ There  are  four  areas  for  "Automation"  that  are 
intimately  interrelated  and  thus  require  a 
mechanism  where  sharing  of  ideas  can  take 
place.  Small  workshops  need  to  be  held  to 
discuss  how  to  encourage  multi-investigator 
proposals. 

♦ The  methodology  for  research  and  development 
in  this  area  needs  to  move  out  from  the  confines 
of  the  laboratory  and  into  real-life  contexts.  The 
field  of  "Automated  excavation"  is  in  an 
exploratory  phase  right  now;  in  situ  style  studies 
must  be  encouraged. 

♦ The  development  of  underground  space  with  its 
multitude  of  application  areas  represents  a 
multibillion  dollar  market  requiring  the  use  of 
automated  excavation  tools  that  are  still  at  a 
prototype  stage  More  than  100  million  dollars 
are  probably  needed  to  complete  ongoing 
research,  requiring  the  active  participation  of  the 
private  sector.  Federal  Agencies  need  to  interact 
with  each  other  and  identify  needed  incentives 
to  attract  private  US  and  foreign  investors. 

♦ Automated  excavation  technology  is  an  integral 
part  of  any  environmentally  sustainable 
development.  However,  there  is  a need  to  define 
the  metrics  for  measuring  sustainability  in 
excavation  procedures.  Macroengineering  may 
be  the  proper  vehicle  to  quantify  sustainability. 
The  creation  of  a "think  tank"  to  address  this 
problem,  is  strongly  recommended. 

♦ The  personnel  to  adequately  handle  the  size  and 
complexity  of  the  new  projects  requiring 
excavation  technology  is  inadequate.  The 
possibility  of  converting  part  of  the  defense 
industry  to  fulfill  the  needs  of  these  major 
government  excavation  projects  must  be  closely 
examined  in  a future  workshop. 
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NIST/FHWA  WORKSHOP  ON 
AUTOMATION  AND  ROBOTICS  IN 
ROAD  CONSTRUCTION,  MAINTENANCE^ND  OPERATIONS 

SESSION  ON  PROJECT-SCALE  CONTROL  AND  DESIGN 


’’SESSION  OVERVIEW:  ISSUES  AND  APPLICATIONS 
IN  PLANNING,  INTEGRATION  AND  DESIGN” 

Arthur  C.  Sanderson 

Road  construction  and  maintenance  require  extensive  coordination  of  workers,  machines, 
and  resources.  The  use  of  advanced  computer  and  automation  technologies  may  provide  means 
to  improve  the  efficiency,  productivity,  and  safety  of  construction  projects.  Often  the  sequence  or 
concurrency  of  operations  is  critical  to  success.  The  complexity  of  planning  and  scheduling  to 
best  achieve  effective  materials  handling,  use  of  machines,  and  quality  of  the  project  is  well-suited 
to  computer-based  methods  which  incorporate  interactive  optimization  and  knowledge-bassed 
planning  tools.  As  more  automated  and  sensor-related  operations  are  developed  for  use  in 
construction  projects,  the  integration  of  machines  and  sensors  becomes  essential.  An 
’architecture'  which  systematically  describes  the  interconnection  and  communications  among 
machines,  sensors,  and  users  makes  this  integration  possible,  and  may  make  many  new  tasks  and 
approaches  possible.  Design  underlies  all  of  the  planning  and  execution  of  projects,  and  new 
computer-aided  design  tools  improve  the  efficiency  of  design,  but  also  yield  a representation  of 
sites,  parts,  and  structures  which  encourage  better  planning, more  complete  integration,  and  the 
possibility  of  new  more  efficient  methods.  'Design  for  automation'  specifically  addresses  the 
development  of  components  and  processes  which  enable  new  operations  and  tools.  This  session 
will  highlight  the  underlying  technologies  for  planning,  integration,  and  design,  and  describe 
examples  of  current  and  future  use  in  road  construction. 
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WHAT  IS  PLANNING? 
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ITDS  Traffic  Data  System  Rathi,  1990 

TRANZ  Traffic  ctlin  work  zones  Faghri  and  Demetsky,  1990 

SAFEROAD  Highway  safety  structures  Roschke,  1991 
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— WALL 


— GIRDER 

— BEAM 

— SLAB 


e_wall 

l_WALL 


— OPENING 

— PILE 


DOOR 

WINDOW 


— FLOOR 

— SPACE 

— FOUNDATION 

— ROOF 

— ELEMENT2  — . 

— ELEMENT3  —I 


OTHER 


(Future  extension) 


Figure  11-3  Elements  of  the  OARPLAN  product  model  in  PMAP  (adapted 
from  Uto  1989]). 


□ Component  properties,  such  as  dimensions,  material  composition,  and 
finish  specifications,  are  provided  when  OARPLAN  accepts  default 
values,  which  are  correct  in  many  cases.  As  project  definition  proceeds, 


section  11./  U ARP  LAN:  The  Object- Action- Resource  Planner 
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OBJECTS  ACTIONS 


COLUMNS  SLABS  BEAMS  ASSEMBLE  TRANSPORT  INSTALL 


Figure  11-5  Reasoning  from  object  relationships  to  activity  dependency  in 
O ARP  LAN. 


Some  specific  examples  of  the  dependency  rules  that  the  OAR? LAN 
prototype  system  utilizes  to  develop  plans  for  low-rise  frame  buildings  are 

□ Supports  constraint  Columns  must  be  placed  before  the  beams  they 
support  The  relation  between  the  object  constituents  of  the  activities 
for  installing  columns  and  installing  beams  is  supported_by. 

Q Safety  constraint  In  steel-framed  buildings,  do  not  start  work  on  the 
members  of  floor  n until  the  slabs  of  floor  n-I  or  floor  n-2  are 
constructed.  The  relation  nere  between  activity  objects  is  that  they 
belong  to  floors  that  are  one  or  two  levels  apart. 
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FICl  RF  13.  Schematic  !avnut  of  j'*  the  kno*  teJjic  Mneks  :n  the  higher-kvcl  module  (From 
Hucnirw>n  P An  t-epen  S>Mem  lof  the  Selevtuft  f Lanh  Rcuianj:  Structures.  M S thesis. 
tniserMis  ct  Swines.  Department  ot  Architecture.  Science.  Australia.  IV8S  With  permission  i 

embankment  or  cut  could  be  constructed  It  not.  then  it  is  determined  b> 
default  that  an  earth-retaining  structure  is  required 

The  knowledge  on  the  types  of  structure  suitable  for  a given  wall  appli- 
cation provides  j higher- level  control  on  the  >earcn  and  determines  the  order 
in  which  the  various  wall  types  are  considered,  and  which  type'  are  considered 
tor  every  application.  If  the  types  considered  by  these  rules  prove  to  he 
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N 


1000  ft 


Waretou* 

Bachplam 

Carpeniry-shop 

Job-office 


1000' 


FIGl’RE  *.  Output  of  CONSITE  after  solving  the  office -building  problem  (From  Hamiani. 
A and  Pope  sen  C Pro*.  5th  ASCE  Coni  Computing  m Civil  Engineering  Microcomputers. 
Will,  k M Ed  . ASCE.  Ne»  \orts.  IVKB.  V*nn  pcrmisMOA  t 
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PROJECT  CONTROL 


QUALITY  SCHEDULING  COST  CONTROL 


INITIAL  IN-PROGRESS 


GENERAL 

REQUIREMENT 

- codes 

- major  subs 


LOGIC  TIME  COST 


procurement 

approvals 


weather 

constraints 


activity 

urations 


- payment  request 

- front-end  loading 


FICl'RE  21.  Knowledge  structure  (Front  O'Connor  M i . Dr  Li  Gvu.  J M . and  Ibtoa. 
C W Jr  . Proc  ASCESymp  Expcn  Syucm*  in  Civil  Engineering.  Koucm.  C N and  Mayer. 
M L . Eds  . ASCE  Seank.  1986.  55  With  pcmuuion  i 

t\pe  of  information  that  contractors  present  to  owners  for  verification  before 
the  commencement  of  the  project.  Typical  information  would  consist  of  the 
owner  s approval  activities,  participation  of  major  subcontractors  in  the  for- 
mulation of  the  plan,  etc  The  in-progress  scheduling  evaluation  module  allows 
project  managers  to  examine  questions  such  as  delay  and  duration  modification 
concerns 

4.3.6. 2.  Methodology 

CONS  AES  (CONstruction  Scheduling  Analysis  Expen  System)  relies 
upon  existing  project  control  system  software  to  (It  identify  and  capture 
expressions  of  similar  form  in  the  "paper-'  knowledge  base.  (2)  determine 
the  specific  target  inference  engine.  (3>  decide  how  the  "paper"  knowledge 
base  is  to  be  represented  in  the  inference  engine,  and  (4)  develop  a mapping 
technique  to  adapt  the  concepts,  facts,  and  rules  to  the  corresponding  engine 
syntax. 
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FIGl  RE  2.  Typical  bndfe  configuration  recommended  by  KYBAS  (From  Fe^ke.  T £ and 
Fenske.  S M Developments  in  Shoe:  j nd  Media*  Span  Bridge  Engineering  90.  Toromo. 
AupuM  tvwi.  23  With  permission  i 


AASHTO  girder  types,  diaphragms  and  their  related  location  in  each  span, 
etc  These  recommendations  are  based  upon  user  input  to  queries  regarding 
clearance,  usage,  and  sue  location  Figure  2 shows  a typical  girder  bndge 
configuration  recommended  by  KYBAS 

The  prototype  RCBD  (Reinforced  Concrete  Bridge  Design)  ES  (Nguyen. 
1*^0 1 for  selecting  a reinforced  concrete  bridge  was  developed  bv  using  the 
VP-E\pen  development  tool  RCBD  is  a rule-based  ES  that  has" more  than 
100  rules  in  its  knowledge  base  There  are  12  different  types  of  bndges  for 
the  goal  variables  and  eight  dependent  variables  for  the  bndge:  span  length. 
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F1GI  RE  2.  Management  tnlurnuiK*)  system.  (From  Howell.  T F . J Tramp  Eng  . 116(6). 
831.  IV90  Wuh  ptmuuwa  ) 

strategies  Transportation  engineers  can  design  and  evaluate  various  alter- 
native traffic  control  strategies  pnor  to  their  real-life  implementation.  This 
evaluation  helps  reduce  motorists'  operating  costs,  vehicular  fuel  consumption 
and  emissions,  and  costly  retrofits,  which  occur  when  a problem  is  detected 
onh  after  implementation. 

Traffic  models,  however,  have  a few  major  difficulties  associated  with 
their  use  Basically,  there  is  no  consistency  in  the  definition  of  traffic -related 
terminology  used  by  different  models  Alvv  since  these  computer  programs 
were  written  for  use  in  a batch  processing  mvKlc.  a card  image  input  data  file 
must  be  prepared  for  each  run.  This  makes  them  awkw  ard  and  time  consuming 
to  use.  Finally,  due  to  the  limitations  of  these  models,  users  who  are  devel- 
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FIGURE  7.  Typical  maintenance  operation  on  seo*ndary  highway  (From  Faghn.  A and 
DemetUy.  M J . J Tramp  lug..  116(h).  IWi  Wish  permi&MOO  i 


■ a 


CO 


UJ 


Q 

O 


UJ 

> 

UJ 

UJ 

h 

UJ 

IT 

O 

CO 

Q 


UJ 

O 


< 

Q 


o 

<o 


o 

o 


■ ■ 


o 


■ MB 

t: 

o 

T3 

C 

CO 

© 

w 

</) 

O) 

c 

■ MB 

c 

c 

CO 


0. 


CO 

3 

O’ 

© 

> 

o 

Q. 


9 

9 

9 

9 

« 

9 

a 

a 

a 

a 

n 

n 

n 

n 

a 

Cl 

i 

n 

a 


PETRI  NET  MODEL 


tj 

i 


CO 


ft 


r 


i 

r 

« 

ft 

ft 

ft 

ft 

ft 

« 

« 


x 


CO 

V) 

(0 

g 

0 £3£?9  MB 

1 O 

0)  £ 

p: 

.£*  o 

< 

.s  « 

< C 

1 Q) 

<o  .2 

cc 

o » 
& 5 

Ul 

C > 

o 

<0  .2 
o « 

. 2 

Ui 

o 0) 

is 

o 

CO  o 

c 

0 3 

<D  Q. 

■ ■■i 

a c 
o c 

2 

o o 

o 

< 

C M 

IH 

(0 

Is 

c 

o 

- 2 

o 

o 8 

E 

2 >* 

o 


CO 


CO  _ 
IAI  O 


w <5  ■o 

w — S 

2 E » 
<0  (o  ,E 

O TJ 

o O CO 

o c ® 

H-  O Q. 
o 4=  (0 

=1  ? 

r — 

col 
"go 


(0 

c 


g> 

= 2 2 

° 2 o 

> Q = 

5 = o 
8 g® 

> 2*^ 
w g (0 

a _ to 

3 O ‘5 

W -D  g 
2 ! §. 
ZIS 

q Z ca 


■D 

gg 

< o 

-1  c 
o o 

II 

o ** 

o ^ 

. i 0) 

3=  W 

«5  3 

■t  0 

■O 

c >» 
(0  (0 

O £ 
£ c 

i| 

o to 

c 
0) 
(0 
® 


5 

o 


0) 

o 


a 

a> 


O 


3 
O 

</)  _ 
Q)  w 

cc  « 


control  traffic  flow  or  coordinate  delivery  of 
materials. 


MULTIMACHINE  INTEGRATION 


0) 

> 

© I 

— - C) 

to 

3 O 

£ " 

o ? 

o ** 

o co 

•rr  #1% 


CO 


0) 


o D) 

fl>  co  .£ 
« 'D  w 

o c C 

O J=  m 

O "8 
*■5  £ 


Q)  CO  _ 

■3-#! 

f Jg 

u O 3 


CO 


CO 

o 

a 


CO  • 

■*  CO 

S g 

O) 

C o c 

CO  TO  £ 

E := 


-ig 

5 co  p 

*-  .Q  0) 

c <o  o 

— © J- 
(0  3 tL 

o •=  c 

isl 

O o.  I 


c2 

2 3 

CO  £ 
o „ 

c g 

Z2 

= CO 

E O 

8 i 

3 I 
£ E 
~ o 
Q o 


c 

o 


co  £ 
c CO 

=6  • 

o? 

o a 
o ~ 

T3  >* 

> c 

o ® 

E 3= 

.=  Q) 


5 o 

CO  is 

7 c 
0)  o 

© o 

C-D 

o c 

£ co 

12 

CO  c 

■2  E 

co  ^ c^- 
u 3 u 

ooo 
£■  >*  c 

° oE 
a>  -o  o 

£ s g 

° x o) 

© c 

O fli'5 

L O O 

fl)  CO  2 
£ X O 

+*  o -e 


CO 

0) 


© 

T3 

© 

oc 


c — 

— © 

a .9- 
E £ 


i 

V 


c 

r 

c 

« 

i 

tf 

i 

Li 

11 

n 

i 

i 

i 


UJ 

o 

z 

< 

9 

O 

> 

< 


c 

o 

■ Mi 

CO 

o 

z 

g 

O) 

c 

■ MB 

c 

o 

i- 

c 

o 

<D 

o 

CO 

0) 

■O 

c 

W 

f 

§ 

Q. 

o 

'co  ® o 

T3 

^ o .£  c 

C5  O <3  -o  o 


O 

c/) 

O 

o 

Q 

z 

< 

z 

o 

H 

O 


CO  M 

S' fell 

1 0,8 

CO  — 
CO  0) 

•=  5 

CD  O 
O Q. 
3 H- 
73  O 


uj 

g 

u. 

LL 

LU 


§1 
T3  3 
CD  7 

£ § 
<0  Jr 

c a 
« E 

Q." 

c2 

° 2 
O 

O 5 

E 

a> 

w 
> 
co 


0)  o 

?s 

a | 

O)  O 

.E  S 

« CD 

£ C 

3 2 

co  a 


m c 

<£  m 
< c 

m §&* 

i T3 

§ a* 

<n  ~ = 


CO 

a 


a. 

UJ 

> 

H 

O 

< 

UJ 

oc 


E o 


uj  or 


a-o 

gS 

Z .=  o 

< 2 O) 

CO  CD  .£ 

z « o> 

< o g 

_J  c ® 


7 o 
o 2 

* o 

CD  2 
CO  o 

3 co 

2 

■D 

C 

CO 


• SAFETY 

Detection  and  avoidance  of  humans  on  the  worksite. 


SITE  INTEGRATION  AND 
HIERACHICAL  CONTROL 


P 

I 

P 

I 

9 

9 

9 

9 

9 

9 

9 

9 

9 

9 

9 

9 

1 

9 

1 


IJ 

i 


1 

( 

r 

H 

f 

r 

i 

i 

e: 

* 

Bl 

1 

» 

tl 

* 


USER3* ► TELEOPERATION  « ► REAL-TIME 

COMPUTING 


DESIGN  FOR  AUTOMATION 


■2  C 
co  o 

0 o 
ra  E 

E* 

1 § 

(O  o 


CO 


r T3 


CO 


£ « 
Q.  O) 

«l 

0)  7 

o c 

> 5 
0)  ^ 

■O  T3 

H-  O 

0 3 

.11 

<0  ts 

a)  E 
Q w 


CO 

s>« 

1 = 

a g 

Is 

CO  c 
CO  « 


o E 


C £ 


3 "O 

c 

3 « 

is  c 
co  .2 

w o 

TJ  CD 
CO  a 

o co 
1-  c 

ll 

.2>o 

8 £ 

Q • 


2 

c 

CD 

E 


> 

c 

o 

JZ 

c 

CO 

VN 

t 

CO 

E 

CO 

vs 

■o 

0) 

o 

a-| 
E g 
■”  0) 

1 1 

C T3 

O)  c 

• 3 

CD  O 
<S(  CO 


TJ 

c 

CO 

c 

o 
• ■■ 

2 

3 

E 

CO 

o 

> 

o 

a. 

E 


CO 

T3 

o 

£ 

o 

E C 

■E*  (0 

2 N 
a>  = 

■O  CO 

5 3 

2 > 


cs 


g 

9 

9 

I 

9 

9 

a 

0 

a 

a 

a 

a 

a 

n 

n 

a 

% 

a 

a 

a 


SPECIAL  CONSIDERATIONS  IN 
D CONSTRUCTION  AND  OPERATIONS 


5 2 

O 3 

■c 

.2  o 

t o 

CD  o 
O a 

o c 

JS  CD 

2 s 

o £ d> 
■■*=  o c 
o > -c 

£ &i 

O § 3 

O w c 

■D  <0  « 

c ■g  E 
w co  co 
d>  § 

c co  © 
c co  E 

C TS  CO 

« I w 

Q."  © 

C 0>~ 

o o 

/«  y r 


CO 

c 

<0  o co 


o 

£ 


■ mm 

CO 

0) 

> 

o 

o 

c 

c 

c 

DM 

« 

a 

O) 

a> 

o 

c 

o 

a mam 

c 

CO 

c 

2 

CO 

o 

c “ <u 
o 32  a 

•.3  C _ 
C <0  3j 
<D  c " 

> O)  2 

L ■“  3 


o co 
o o> 

Is- 

XI  <D 

CO  .2 
T3  tt 
C 


CO 

c 

o 

E 

® 

T3  . 

0)  Jt 
a 3 

M CO 
3 CO 


ppliers  has  not  been  the  traditional  mode  of 
eration.  New  incentives  for  coordination  and 
egration  may  be  necessary. 


SUMMARY  OF  OPPORTUNITIES 


TO 


V) 


O) 

TO 


O) 

c 


c 

c 

TO 

a 


w 

E <o 


£ w 


0) 


--T*  TO 
V)  £ 

-a  a 
to  +* 

W (0 
TO  O 

■9  « 

TO  "O 
O)  C 
T3  (0 
TO  ** 


<5  3 


c 

o 


■TO  TO 

TO  2 


■o  w ® 


5 ® 

TO  C 

C£  ® 

o o 

#5  TO 
TO  E 


E c 


o 

TO 

E 


c 

? ° 

O OSS 
O C 3 

o=  E 


£ o 


»■§ 


•jr  TO  — 
w ]=  TO 


i Ol 
P <0  TO 


TO  O)  - 

TO  Q)  E 

■O  „ •*- 

o O)  o 

E C TO  _ 

; 2 « ■=  o 

TO  *-  'C  ^ 

C O TO 


° 3 • 

III 

TO  O iE 

2 o o 

<?T3  TO 

TO  S £ 

w 

TO  13 
TO  JQ  s 

.■Sira 
« O S 

g & 

o o 

TO  TO  TO 
TO  . TO 

o>~ 


a 

TO 


o 

TO 


O TO) 
. C 

TO  '-5 

•S  2 

“ TO) 


> 

mmam 


TO 

O 


o o 


•=  I = 

^ #1%  "m 


| is  TO  c -S  ® - P 


__  E 

TO  g 
TO 


O) 

C 


5 

o 


x ^ 


c 

O) 


TO 
TO 
P T3 


« O)  • 

g»o 

f £S 

° TO  C 

•-TO  £ 
TO  TO  C 
TO 

^ ^ TO 

s g £ 

TO  ~ "C 
C >.  C 
TJ  g TO 
c S c 

Q .2  o 

§ gf 

I » 8. 

1*1 

3 3 0 
TO  O-  **” 

O "O  2 
° ® 8) 

0)1? 


5)  Q--° 


S.i*s 


I 

0 

I 

u 

i 

9 

9 

9 

9 

9 

n 

n 

n 

n 

i 

h 

« 

A 

A 


National  Institute  of  Standards  & Technology 
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Figure  1.  The  NIST  robot  crane.  An  octahedron  structure  with  an  equilateral 
triangle  at  the  top  supports  a work  platform  that  can  carry  various 
tools  or  equipment.  The  work  platform  is  suspended  by  six  cables 
that  are  controlled  by  six  winches. 
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Automation  Integration  for  Construction 


A.  B.  Cleveland,  Jr. 

President 

Jacobus  Technology,  Inc. 

Gaithersburg,  MD  20879 

Abstract 

When  compared  to  engineering  design,  the  use  of  automation  in  construction  is  still  fairly 
limited.  The  applications  of  automation  to  construction  have  largely  been  focused  in  the 
general  area  of  project  management  for  activities  such  as  scheduling,  manpower  tracking, 
billing,  materials  control,  etc.  Since  the  advent  of  powerful  personal  computers,  these 
technologies  are  routinely  used  at  construction  sites.  However,  with  the  increased  use  of  3D 
modeling  in  the  design  process,  coupled  with  the  increased  availability  of  cost  effective 
hardware  and  software,  there  are  now  many  opportunities  for  strategically  new  applications 
of  automation  to  construction.  In  this  discussion,  we  will  focus  on  two  very  specific 
examples  of  new  uses  of  automation.  Both  of  these  applications  are  currently  underway  for  a 
major  transportation  projection.  The  first  application  is  the  integration  of  project  schedule 
information  with  3D  animation.  This  application  makes  it  possible  to  see  and  compare 
various  schedule  alternatives  in  a 3D  real-time  animated  mode.  The  second  application  is  the 
extension  of  3D  real-time  animation  into  the  area  of  virtual  reality.  In  this  case,  virtual 
reality  is  being  tested  to  assess  its  viability  for  reviewing  transportation  designs,  such  as 
automobile  tunnels,  by  simulating  the  experience  of  the  car  driver.  Both  of  these 
applications  are  good  examples  of  potential  uses  of  automation  for  construction  which 
leverage  electronic  data  generated  during  the  design  process. 

Biography 

Mr.  Cleveland  is  president  of  Jacobus  Technology,  Inc.  in  Gaithersburg,  Maryland.  Jacobus 
Technology  provides  consulting  services  and  software  products  to  the  Architecture, 
Engineering  and  Construction  (AEC)  industry,  specializing  in  the  area  of  computer-aided 
design,  computer-aided  engineering  and  animation. 

Prior  to  this,  Mr.  Cleveland  was  the  Manager  of  Automation  Technology  for  Bechtel 
Corporation  in  Gaithersburg,  MD.  He  was  responsible  for  hardware  and  software 
procurement,  systems  operations,  networking,  user  support,  software  development  and 
computer-aided  design  activities  in  the  Gaithersburg  Regional  Office.  He  supervised  the 
start-up  of  the  group  in  1980  and  directed  the  development  of  Bechtel’s  3-D  plant  design 
system,  3DM™,  and  the  real-time  animation  system,  Walkthru™.  Mr.  Cleveland  was 
appointed  as  a Bechtel  Fellow  in  1987. 

He  graduated  from  Johns  Hopkins  University  in  1972,  where  he  received  a Bachelor  of 
Engineering  Science  degree  in  Operations  Research  and  Industrial  Engineering.  He  has 
authored  over  25  publications  concerning  computer-aided  design  and  engineering. 
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Background 

• Engineering  automation 
* Engineering  analysis 

► Computer-aided  drafting 

- Computer-aided  design  and  engineering 

• Incentives  for  engineering 

► Reduced  cost  (fewer  hours) 

► Shorter  schedules 


Jacobus  Technology,  Inc. 


Background 

• Construction  automation 

► Construction  schedules 
- Status  databases 

► Material  inventory 

• Incentives  for  construction 

► Reduced  field  engineering  hours 

► Improved  materials  management 


Consequences 

• Focus  at  organizational  level  only 

• Unable  to  leverage  automation  efforts 
of  other  organizations 

• Limits  to  potential  benefits 

• Total  constructed  cost  not  addressed 
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Schedule  Animation 

• Background 

► Complex,  multi-year  construction  project 

- Multiple  design  contractors 

► Multiple  construction  contractors 

- Logistics  during  construction  critical 
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Schedule  Animation 

• Solution 

► Develop  3D  models  from  design 
contractors 

► Integrate  design  models  with  schedule 
data 

* Simulate  construction  with  real-time 
animation 


Construction  Simulation  Toolkit 

• Integrates  project  schedules,  3D 
models,  and  real-time  animation 

• Displays  project  being  'built'  per 
project  construction  schedule 

• Animates  multiple  alternatives 

• Equipment  library  for  simulation 
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Construction  Simulation  Toolkit 

• Software  architecture 


Construction  Simulation  Toolkit 

• Benefits 

► Detect  schedule  problems  prior  to  the 
start  of  construction 

► Improve  communication  and  ensure 
consistent  interpretations  of  schedule 

- Quickly  evaluate  multiple  alternatives 

► Improved  control  via  better 
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Virtual  Reality 


Background 

- New  tunnel  alternative 

► Questions  regarding  drivers'  perceptions 

- Unable  to  evaluate  design  with  current 
tools 


Virtual  Reality 

• Solution 

► Use  existing  3D  computer  models 

► Simulate  driver's  path  through  the  tunnel 

► Display  image  via  eyephone  headset 


Jacobus  Technology,  Inc. 


Where  is  this  leading? 

• Use  of  automation  will  no  longer 
be  driven  by  local  needs,  e.g., 
construction  will  determine  the  need 
for  3D  models 

• More  focus  on  total  constructed  cost 

• Facility  operation  is  next 


Technology,  Inc. 


What  do  we  need? 

• Technologies 
- Object-oriented  systems 
•/  More  robust  data  representations 
'f  More  flexible  systems 
► More  applications  for  the  jobsite 
v'True  construction  applications 
v'  Designed  for  the  casual  user 


What  do  we  need? 

• Cultural  changes 

► Incentives  for  integration 

v Focus  on  constructed  cost 
''Will  require  new  metrics 

► Sensitivity  to  difficulty  of  changing 

v People  do  act  in  their  best  interest 
v People  who  seem  the  most  stubborn 
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Conclusion 

• Computer-integrated  construction  is 
a continuous  evolutionary  process 

• Incentives  for  automating  and 
integrating  are  dynamic  and  will 
change  as  work  processes  change 

• We  must  give  people  an  incentive  to 
automate 
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Large  Scale  Robotics 


Chaired  by: 

Kerien  Fitzpatrick,  Carnegie  Mellon  University 


Participants: 

Charles  Schaidle,  Caterpillar,  Inc. 
Anthony  Stentz,  Carnegie  Mellon  University 
William  Whittaker,  Carnegie  Mellon  University 
James  Albus,  NIST 


Ronald  Lumia,  NIST 
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Technologies  for  Automated  Earth-Moving,  Spreading, 
Compacting,  Lifting  and  Positioning  of  Materials  and 

Structural  Elements 


Kerien  Fitzpatrick 
Carnegie  Mellon  University 
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Background 

Automation  is  a by-product  of  progress.  As  a specific  process  evolves,  its  needs  become 
well  understood  and  predictable.  This  progression  toward  automation  revolutionized 
the  manufacturing  world  when  robotic  manipulators  and  mechanized  plants  demon- 
strated what  is  achievable  once  the  appropriate  equipment  exists  for  the  task(s)  at  hand. 
Productivity  boomed,  quality  increased  by  orders  of  magnitude,  and  costs  dropped. 
Critics  argue  that  manipulators  and  equipment  used  in  manufacturing  are  not  robotics, 
but  mechanization,  and  the  lessons  learned  are  not  applicable  to  arenas  such  as  road  con- 
struction, farming,  and  forestry. 

This  distinction  is  not  clear  since  the  basic  principles  and  technologies  required  to 
automate  are  the  same.  Road  construction  could  be  considered  manufacturing  on  the 
move.  Its  perfect  equipment  moves  over  the  path,  absorbs  materials,  and  lays  down  a 
complete  roadway  of  high  quality  and  low  cost.  The  technologies  of  perception, 
planning,  control,  and  site  integration  used  in  manufacturing  are  based  upon  the  same 
algorithms  and  techniques  as  those  needed  for  arenas  such  as  road  construction.  The 
prime  differences  between  manufacturing  and  an  outdoor  task  such  as  road  construction 
include  the  following. 

Structure  - Manufacturing  is  optimized  for  structure,  components  come  down  a 
certain  belt,  are  modified  in  some  way,  and  shipped  off  on  another  belt.  Most 
equipment  performs  a single  operation.  Construction  is  less  structured  than 
manufacturing,  but  is  still  presents  significant  structure.  Materials  can  be 
unloaded  in  specific  areas  to  sort  by  task  and  designs  can  be  modified  to  suit 
automation.  All  structure  existing  in  manufacturing  did  not  occur  naturally,  it 
was  created  - the  same  principle  can  be  applied  to  construction  automation. 

Mobility  - Manufacturing  is  optimized  by  fixing  the  location  of  equipment  while  in 
construction  everything  is  on  the  move.  The  difference  is  construction 
automation's  additional  need  for  position  estimation  and  registration.  For 
example,  a bulldozer  might  be  considered  a milling  machine  that  moves.  All  it 
needs  to  know  is  its  position,  or  pose  when  you  consider  the  many  actuators  on 
construction  equipment,  and  have  the  ability  to  inform  other  nearby  or 
cooperative  equipment.  In  past  years  this  goal  was  unachievable  since  the 
accuracy  necessary  (a  couple  of  inches  or  less)  required  very  expensive 
equipment.  Cost  of  recent  generations  of  commercial  positioning  equipment 
(lasers  and  global /satellite  positioning  systems)  capable  of  achieving  the 
necessary  accuracy  has  dropped  significantly.  This  accurate,  reliable,  and 
inexpensive  position  information  will  revolutionize  the  usage  of  the  class  of 
equipment  used  in  highway  construction  and  maintenance. 

Multi-Duty  - Manufacturing  benefits  from  configurations  that  minimize  the  need  for 
equipment  to  perform  multiple  tasks.  In  construction,  backhoes  and  bulldozers 
perform  many  different  tasks.  This  does  not  preclude  automation  for  certain 
costly  or  dangerous  tasks  such  as  trenching  and  pipelaying. 

Site  Integration  - While  manufacturing  benefits  from  simple  communications  paths, 
generally  predictable  materials  delivery,  and  stable  resources,  extrapolating  the 
techniques  used  in  this  arenas  to  serve  construction  is  quite  viable.  Construction 


projects  can  benefit  significantly  by  implementing  existing  techniques  with  little 
change.  In  fact,  current  construction  projects  perform  planning,  scheduling,  and 
resource /materials  control,  but  the  difference  is  that  they  are  more  disjoint  at 
certain  levels  (multi-contractors). 

Thus,  the  technology  advancements  necessary  to  make  highway  construction,  mainte- 
nance, and  operations  feasible  continue  to  occur  in  one  form  or  another  in  different  areas 
of  robotics  research.  To  enable  these  advancements  to  benefit  FHWA  interests,  they  must 
be  applied  to  characteristic  or  archetype  tasks  to  further  the  understanding  required  for 
successful  automation.  Indeed,  some  equipment  manufacturers  have  recognized  this 
potential  and  are  investigating  certain  enhancements  such  as  "intelligent"  machinery 
extending  to  automated  haulage  for  surface  mining.  It  is  in  the  FHWA  interests  to  guide 
these  developments  by  directing  research  into  core  areas  which  show  the  greatest  prom- 
ise of  success. 


Rationale 

To  identify  which  archetype  tasks  should  be  more  fully  explored,  the  panel  evaluated 
their  potential  based  upon  current  technology  levels,  current  benefits,  long-term  bene- 
fits, and  risk.  Four  key  areas  were  itemized  to  represent  the  basic  components  of  technol- 
ogy relevant  to  highway  construction,  maintenance,  and  operations.  These  areas  include 
sensing  and  inspection,  large  scale  robotics,  teleoperation  and  human  interface,  and  inte- 
gration design,  plan,  and  schedule.  Each  technology  area  is  then  subdivided  to  even 
greater  detail  which  are  represented  in  Figure  la.  Construction  and  maintenance  were 
evaluated  as  the  greatest  opportunities  for  automation.  The  two  flow  charts  in  Figure  2 
depict  the  process  through  which  a project  comes  to  fruition.  Central  to  all  tasks  are  plan- 
ning and  scheduling,  indicating  that  improvements  in  this  area  would  definitely  benefit 
the  FHWA. 


Construction 


Maintenance 


PL/VN' 


Schedule 


PLA.N 


Schedule 


Construction 

Construction  topics  inherently  provide  an  excellent  opportunity  for  automation.  Much 
of  this  is  due  to  the  nature  of  the  project.  Construction  projects  take  over  a specific  area 
and  adapts  the  area  to  the  needs  of  the  project.  Since  adaptation  is  already  performed, 
changes  to  existing  methods  would  not  be  as  traumatic  as  implementing  an  entirely  new 
scheme.  The  changes  for  automation  could  be  represented  as  evolution  instead  of  revo- 
lution. Three  areas  were  identified  as  prime  candidates  for  automation  using  the  criteria 
listed  in  the  Rationale  section  - site  integration,  bridge  deck  construction,  and  trenching 
& pipelaying.  Each  topic  is  discussed  briefly  in  the  following  paragraphs.  Greater  detail 
can  be  found  in  the  proposal  sections. 

Current  methodology  separates  an  overall  project  into  multiple  sites,  each  having  a spe- 
cific site  for  which  they  are  responsible.  Each  site  has  its  own  plan  and  schedule  within 
the  overall  project  schedule.  A site  plan  can  be  further  divided  into  plans  and  schedules 
for  the  individuals  tasks  required  to  complete  the  job.  This  organization  method  identi- 
fies site  integration  which  includes  site  design/  planning,  scheduling  as  a key  focal  point. 
Improvements  in  site  integration  provide  direct  benefits  to  the  contractors  involved,  the 
DOTs,  and  to  FHWA.  Details  of  the  actual  benefits  are  presented  in  the  first  project  titled, 
"Site  Integration  Through  Hierarchical  Control."  This  topic  would  serve  to  develop  the 
extensions  to  site  integration  listed  in  the  Background  section. 

Evaluation  of  tasks  sitting  below  site  integration  identified  a small  number  that  already 
exhibited  significant  structure  in  the  task's  environment.  Automation  projects  such  as 
these  are  less  costly  and  have  less  risk  since  the  automated  method  can  take  advantage 
of  the  existing  structure.  Certain  bridge  deck  designs  and  construction  methods  offer  an 
excellent  opportunity  for  automation.  The  screed,  commonly  used  for  bridge  deck  con- 
struction, represents  an  existing  structure  which  could  be  modified  to  perform  addi- 
tional duties  beyond  the  spreading  and  finishing  of  concrete.  The  proposals  on 
automated  bridge  deck  construction  and  rebar  installation  provide  examples  of  what  is 
feasible  using  a modified  screed  or  its  support  structure.  Work  of  this  type  precludes  the 
need  for  position  estimation  and  registration  of  mobile  equipment,  as  mentioned  in  the 
Background  section  and  places  emphasis  on  the  advantages  of  structure. 

The  last  of  the  construction  applications  is  that  of  trenching  and  pipelaying.  The  primary 
drivers  for  this  task  are  safety  and  efficiency.  This  area  requires  additional  support  due  to 
its  mobility  which  increases  the  complexity  of  automation.  However,  it  is  an  ideal  first 
step  since  the  process  has  the  equipment  move  from  location  to  location  and  the  station- 
ary time  at  each  location  is  sufficient  for  many  cost-effective  position  estimation  and  reg- 
istration methods.  This  proposed  topic  also  addresses  the  need  to  automate  multi-duty 
equipment  such  as  a backhoe. 

Maintenance 

Maintenance  differs  from  construction  in  that  the  task  or  project  may  not  be  of  significant 
duration,  in  fact,  the  duration  might  even  be  less  than  a full  shift.  While  the  setup  cost  for 
maintenance  may  not  rival  that  for  construction,  all  DOTs  visited  agreed  that  in  the 
future  the  majority  of  budgets  would  be  directed  towards  maintenance.  This  places  addi- 
tional requirements  on  automated  systems  such  that  their  intrusion  into  the  traffic  flow 


be  limited.  When  a maintenance  project  reaches  such  proportions  that  roadways  or 
bridges  are  shutdown  for  length  periods  of  time  - they  resemble  more  of  a construction 
project  and  can  be  treated  as  such.  Using  the  same  criteria  as  construction,  two  topics 
were  identified  as  having  significant  relevance,  near  and  long  term  benefits,  and  mini- 
mum risk  given  existing  technology. 

The  first  topic,  bridge  inspection  and  maintenance,  incorporates  mobility  and  structure 
in  the  near  term  with  the  potential  for  multi-duty  in  future  years.  Initial  versions  would 
most  likely  take  advantage  of  existing  machinery,  such  as  snooper  trucks,  to  minimize 
the  cost.  Thie  human  bucket  would  be  replaced  by  custom  tooling  to  handle  a specific 
task.  In  future  years,  the  snooper-based  machinery  might  be  replaced  by  optimized 
machinery  which  would  attach  to  the  bridge  and  would  feature  a variable  toolset.  The 
proposal,  "Automating  Bridge  Inspection  and  Maintenance"  provides  further  details  of 
how  a system  would  benefit  the  FHWA. 

The  second  topic,  "Pavement  Inspection  and  Repair,"  focussed  on  improving  the  inspec- 
tion and  evaluation  process  used  in  highway  maintenance.  To  limit  intrusion,  this  pro- 
cess must  occur  at  highway  speeds.  This  speed  requirement  places  significant  emphasis 
on  mobility  and  structure  due  to  the  limited  visual  presentation  offered  by  the  defects. 
Ideally,  detecting  defects  after  they  become  visual  may  not  suffice  in  the  future  and  thus 
the  sensing  may  have  to  detect  them  prior  to  becoming  visual.  Beyond  the  actual  detec- 
tion problem,  another  problem  of  magnitude  must  be  overcome.  Sampling  for  the  small 
defects  for  an  entire  lane  at  highway  speeds  also  requires  improvements  in  data  acquisi- 
tion modes.  The  data  rate  can  easily  exceed  gigabytes  per  second  if  filters  or  compression 
are  not  used.  The  proposed  project  outlines  a method  to  reduce  the  magnitude  of  the 
data  collected  and  a scheme  to  overcome  the  mobility  problem. 


Earthmoving  in  the  Information  Age 


C.L.SchaidIc 
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ABSTRACT:  Computers  and  communications  technology  have  revolutionized  many  industries. 
In  earthmoving  for  highway  construction,mining  and  site  preparation,  this  revolution  has  just 
begun.  The  major  changes  are  still  to  come,  but  they  are  just  around  the  comer.  This  pesentation 
is  a vision  of  this  up-coming  revolution  in  our  industry.  This  vision  is  comprehensive.  It  includes 
basic  communication,  machine  monitoring  and  diagnostics,  job  and  business  management, 
planning  and  operations,  and  machine  control.  For  our  industry  to  realize  maximum  benefits 
from  the  information  age,  we  must  have  a vision  of  the  future  that  is  shared  by  technologists, 
machinery  producers,  end  users,  and  job  planners.  Using  a highly  visual  format,  this  presentation 
describes  such  a vision. 


THE  VISION 

• Basie  Communications 

• Monitoring 

• Management 

• Planning  / Operations 

• Control 

• Total  Information  System 


| KEY  TECHNOLOGIES  | 
Existing 

Low  Cost.  Powerful  Computing 
Glooal  Communications 
Software 

Sensors.  Actuators.  Displays 

Under  Development 

Hi  SDeeo.  Accurate  Locating 
Low  Cost.  Accurate  Mapping 
Lower  Cost  Sensor,  Actuators.  Displays 
More  Application  Specific  Software 
Systems  Integration 
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DIGITAL  SITE  INPUT 


EABTHMOVING  INC. 


• Checker 
• Inspector 

• Grade  Man 

• General  Foreman 

• Earthmoving  Foreman 

• Surveyor 


• Systems  Coordinator 

• Communications  Manager 


Machines  / Operators  interlacing 
With  People  lor  Information 


Machine  / Operators  Interlacing 
with  Digital  World  for  information 


KEY  CUSTOMER  VALUES 


Reduced  Human  Effort 
Reduced  Material  Usage 
Improved  Job  Quality 
Better  Documentation 
Faster  Payment 
Fewer  Mistakes 
Higher  Machine  Utilization  , 


LOWER 

COST 


Robotic  Excavation  of  Buried  Objects 
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Geomechanical,  Geotechnical  and  Geo-Environmental  Systems 
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Excavation  Scenario 


Depth  in  the  sand  (cm) 


Processed  GPR  image  of  three  objects 
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Depth  in  the  sand  (cm) 


Raw  GPR  image  of  three  objects 
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An  Example 


Terrain  to  be  Excavated 


Perception  System 


INPUT:  Sequence  of  range  images 
from  Perceptron  laser  scanner. 


OUTPUT:  Terrain  elevation  maps. 
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Maps  built  on  demand  from  other 
modules  (planners  etc.). 
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Vohun*  Constraint  (Loader) 


Shaping  CoaitraiBt  (Loader) 


Simulation  of  Loading 


Force  Construct  (Leader) 


Current  Accomplishments 


Developed  perception  software  to  build  elevation  maps  from 
laser  radar  data,  fit  tool-soil  models  to  force/torque  data,  and 
detect  and  localize  small  buried  objects  using  ground 
penetrating  radar  data. 

Developed  planning  software  to  select  optimal  digging 
strategy  based  on  initial  soil  shape,  manipulator  constraints, 
and  target  soil  shape. 

Completed  an  excavation  testbed,  including  Cincinnati 
Milacron  T3  manipulator  with  bucket,  force  sensor,  ground 
penetrating  radar,  laser  radar,  and  sandbox  for  validating  our 
approaches. 


Human-Machine  Interface 


Control  Modes 


Display  Types 


Task 

Control 

Labelled 

Display 

Volume 

3-D  Subsurface 

Control 

Shape 

Dig  Cycle 

3-D  Surface 

Control 

Shape 

End  Effector 

Force 

Control 

Feedback 

Direct  Joint 

Live  Video 

Control 

Program  Goal  and  Status 

Objective: 

• Develop  and  integrate  perception,  planning,  and  manipulation 
technologies  necessary  for  autonomously  detecting, 
uncovering,  and  retrieving  fragile  buried  objects. 

Applications: 

• Space  exploration  and  data  sample  acquisition 

• Characterization  and  remediation  of  hazardous  waste  and 
unexploded  ordnance 

• Mining  and  construction: 

• detection  and  unearthing  of  buried  pipes 

• debris  removal 

• trenching 

• pipe  laying 


Field  Robots  for  Construction,  Maintenanc 
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Field  Robots  for  Construction,  Maintenance,  & Inspection 
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Site  Integration 

through 

Hierarchical  Control 


Jim  Albus 
Ron  Lumia 


Robot  Systems  Division 

National  Institute  of  Standards  and  Technology 
Gaithersburg,  Maryland 
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SITE  INTEGRATION 


A construction  site  is  like  a symphony  orchestra 
with  many  different  players  and  different  skills. 


If  all  the  instruments  play  their  assigned  tasks  (notes), 
at  the  proper  time  (beat),  the  result  is  a beautifully 
coordinated  activity  (music). 


If  the  players  do  the  wrong  tasks,  at  the  wrong  time, 
confusion  results  (dissonance). 
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SITE  INTEGRATION 


A plan  is  like  a musical  score. 

Each  player  has  a part  to  play. 

The  parts  are  synchronized  to  form  harmony. 

When  the  plan  is  right  and  is  properly  carried  out, 
everything  arrives  just  in  time, 
at  the  right  place, 
in  the  right  amount 

Everything  fits  properly. 

Nothing  is  lost  or  late. 

Nothing  requires  rework. 

Nothing  is  damaged  or  wasted. 
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SITE  INTEGRATION 


Unfortunately,  a construction  site  is  not  exactly 
a symphony  orchestra.  It  is  not  even  like 
manufacturing. 

Just  in  time  delivery  is  hard  to  achieve. 

TrafTic  causes  delays. 

Things  don't  always  fit 

Tight  tolerances  are  hard  to  hold  in  the  field. 

Construction  sites  are  messy  and  dirty. 

It  rains,  things  get  lost  and  broken. 
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PROPERTIES  OF  HIERARCHICAL  CONTROL 


Limit  complexity  and  span  of  control 
Divide  and  conquer 

Add  capabilities  incrementally  from  manual, 
to  teleoperation  (fly-by-wire), 
to  supervisory  control, 
with  growing  autonomy. 

There  exist  merging  technologies  for 
design  and  system  engineering. 
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EXAMPLES  OF  HIERARCHICAL  CONTROL 

Ancient  Chinese  government  (3000  B.C.) 
Military 

Business  and  government 

Factory  control,  machine  tools,  robots 

Intelligent  control  of  space  robotics,  unmanned 
ground  vehicles,  unmanned  air  vehicles, 
undersea  vehicles, 
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INTELLIGENT  CONTROL  ARCHITECTURES 


Blackboard  • Stanford 

SOAR - CMU 

Subsumption  - CMU 

Task  Control  Architecture  • CMU 

Pilot's  Associate  - DARPA 

Autonomous  Land  Vehicle  - DARPA 

Intelligent  Task  Automation  • AF,  DARPA 

Real-time  Control  System  (RCS)  - NIST 

NASREM  - NASA/NIST 

Meystel  - Drexel 

Saridis  • RPI 

U.  New  Hampshire 

RIPE  / RIPPLE/GISC  - Sandia 

Next  Generation  Controller  • AF,  NCMS 

etc.,  etc., . . . 
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AN  ARCHITECTURE  FOR  SITE  INTEGRATION 


The  NIST  Real-time  Control  System  (RCS)  is  a 
hierarchical  control  architecture  that  can  integrate 
many  people  and  machines  into  a coordinated  system. 


RCS  provides  an  engineering  methodology  for 
designing  hierarchical  control  systems. 


RCS  defines  how  global  plans  can  be  decomposed  into 
local  actions  that  can  be  produced  by  servo  motors, 
hydraulic  flow  rates,  and  engine  power  profiles. 

RCS  deals  with  planning  horizons  ranging  from  years 
and  months,  down  to  jobs  lasting  weeks,  days,  and  hours, 
and  tasks  lasting  minutes,  seconds  and  milliseconds. 
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PRIOR  APPLICATIONS  OF  RCS 


Vision-guided  robot  manipulation 

Machine  tool  (unloading 

Workstation  control 

Robot  deburring 

Turbine  blade  chamfering 

Composites  fabrication 

Automated  Manufacturing  Research  Facility 

Cell  control 

Unmanned  land  and  undersea  vehicles 
Coal  mine  automation 
Submarine  operational  automation 
Space  station  telerobotic  servicer 
Next  generation  controller 
Next  generation  inspection  system 
Vision-based  highway  driving 
RoboCrane 
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ELEMENTS  OF  RCS 
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SITE  METROLOGY 

Registration  of  plans  with  site  landmarks 
Excavation  and  grading  to  plan 
Placement  of  forms  and  objects 
Positioning  of  machines  and  tools 
"As-built"  vs.  "As-designed"  database 
Cut  to  fit 

Hold  tolerances  with  in-process  measurement 
and  feedback  control 
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BENEFITS  OF  SITE  INTEGRATION 


Systems  can  communicate  up-to-date  information 

Sensors  can  detect  errors  and  variable  conditions 

Materia]  delivery  schedules  and  equipment  use 
plans  can  be  regenerated  in  real-time 

Operators  can  have  complete  information  - where, 
when,  how  much  - with  visual  displays 

Novice  operators  can  produce  skilled  performance 

Measurements  can  be  made  (semi)automatically 

Required  dimensions  can  be  recomputed 
in  real-time  for  "as-is"  conditions 
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At  what  cost?  Is  it  economically  feasible? 

Will  it  save  money? 

Under  what  ground  rules? 

What  incentives? 

Short  term  & local 

(next  quarterly  report,  individual  firm) 
vs. 

Long  term  & global 

(3-30  years,  national  infrastructure) 
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SITE  INTEGRATION 


It  has  not  been  practical  to  organize  a construction  site 
like  a factory  — until  now. 

Computer  and  communications  revolution 


High  Performance  Computing  and  Communications 
(HPCC) 


CAD,  process  planning,  expert  systems,  intelligent 
control,  learning,  sensors,  sensory  perception, 
distributed  databases,  knowledge  representation, 
part  geometry  and  attributes,  data  exchange  standards, 
supervisory  control,  operator  interfaces,  virtual  reality. 
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SUGGESTED  NEXT  STEPS 

Pick  a construction  project 

Design  and  install  a site  metrology  system 

Design  and  implement  a communication  network 
and  distributed  database  system 

Develop  a real-time  materials  delivery  scheduler 

Introduce  machines  with  computer-assisted  control 
and  advanced  operator  interfaces 

Measure  performance  and  test  alternatives 
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Session  #3 


Teleoperation  and  Human  Interfaces 


Chaired  by: 

Leonhard  Bernold,  North  Carolina  State  University 


Participants: 

Antal  Bejczy,  Jet  Propulsion  Laboratory 
H.  Lee  Martin,  Telerobotics  International,  Inc. 
H.  Mcllvaine  Parsons,  HUMPRO 


NSF/NIST/FHWA  WORKSHOP 


on 


"Research  Needs  in  Automated  Excavation 
and  Material  Handling  in  The  Field" 


Session  3:  Teleoperation  and  Human  Interfaces 


CHAIRPERSON:  Leonhard  E.  Bemold 

North  Carolina  State  University 


Participants: 

Leonhard  E.  Bemold,  Construction  Automation  and  Robotics  Laboratory,  NCSU 
Antal  K.  Bejczy,  Jet  Propulsion  Laboratory 
H.  Lee  Martin,  TeleRobotics  International,  Inc. 

H.  Mcllvaine  Parsons,  Human  Resources  Research  Organization  (HumRRO) 


Technical  Description: 

Smart  tools,  adaptive  control  of  heavy  manipulators,  data  sensing  and  data  representation 
for  teleoperated  machines,  opportunities  through  "virtual  reality",  lessons  learned  from 
the  application  of  remote  manipulation  systems  in  nuclear  environments,  design  aspects 
for  human-machine  interfaces. 


LOCATION:  Gaithersburg,  Maryland 


DATE:  April  28-30,1993 


s 

p 

p 

p 

p 

Li 

« 

P 

[1 

■ 

P 

P 

[« 


Teleoperation  and  Human  Interfaces 
Technical  Description: 

The  development  of  fully  automated  and  robotic  machines  for  construction  has  to 
address  many  different  but  interrelated  issues.  The  four  main  elements  include: 
electronic  sensing,  intelligent  control,  human-machine  interfaces,  and  mechanical 
adaptations.  These  and  many  other  pieces  of  this  large  "puzzle"  need  to  be  considered  in 
a "holistic"  manner  since  they  are  very  closely  interlinked. 

A teleoperated  machine  is  considered  a remotely  operated  system  that  is 
controlled  by  (a)  human  operator(s)  in  real-time  (or  on-line). 

This  session  will  address  four  aspects  of  teleoperation  presented  by  experts  from 
different  industries/backgrounds  namely:  1)  Smart  tools  and  adaptive  control  in 
construction,  2)  Large  volume  manipulation,  3)  Control  and  information  issues  in 
teleoperation  mode,  and  4)  Design  of  human-machine  interfaces. 
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SMART  TOOLS  AND  ADAPTIVE  CONTROL  FOR  CONSTRUCTION, 

by 

Dr.  Leonhard  E.  Bemold,  Director,  Construction  Automation  and  Robotics  Laboratory, 
North  Carolina  State  University,  Raleigh,  NC  27695-7908 


Although  smart  tools  don't  fit  the  definition  of  a teleoperated  machine,  they 
represent  a category  of  construction  "equipment"  that  merged  some  important  elements  of 
advanced  technologies  with  traditional  (old)  technology.  For  example,  a smart  nailer 
increased  labor  productivity  when  used  by  less-skilled  laborer.  The  advantages  of  this 
innovative  device  over  traditional  nailers  stem  from  the  use  of  an  electronic  sensor, 
directly  attached  to  the  nailer,  which  was  able  to  inform  the  worker  real-time  when  the 
nailer  head  was  lined  up  with  the  (invisible)  stud  behind  the  plywood.  Thus,  nailing 
accuracy  was  increased  and  time  for  measuring  and  marking  the  position  of  studs  was 
eliminated. 

Several  aspects  of  construction  operations  create  great  challenges  for  the 
development  of  robust  and  reliable  controls.  Some  to  the  important  facts  are: 

1)  Construction  takes  place  in  a generally  unstructured  environment,  2)  the  tasks  to  be 
performed  require  mechanical  contact  with  many  non-homogenous  materials,  and  3)  the 
materials  to  be  manipulated  are  generally  heavy  and  some  important  ones  chip  or  break 
when  handled  without  care  (e.g.,  concrete  elements). 

Excavation  of  soil  has  many  characteristics  that  represent  the  challenges  of 
construction  robotics.  It  can  be  considered  one  of  the  most  unstructured  problems  in  the 
construction  environment  On  one  hand,  many  technologies  have  been  developed  over 
history  to  build  shelters  using  earth.  On  the  other  hand,  the  combinations  of  encountered 
soil  characteristics  are  almost  infinite  and  may  vary  significantly  within  the  range  of  one 
meter.  While  many  traditional  control  strategies  developed  for  applications  in 
manufacturing  can  not  be  used  for  controlling  an  excavator  boom  (e.g.,  position  control), 
a more  sophisticated  model,  depicted  adaptive  control,  has  been  found  useful.  An 
adaptive  controller  is  able  to  process  real-time  information  about  the  environment  during 
operation  and  to  adjust  (adapt)  their  behavior  to  the  changing  environment.  The 
impedance  control  model,  in  particular,  stresses  the  inclusion  of  the  environment  into  an 
overall  control  strategy.  It  considers  the  dynamic  relationship  between  the  actuator  and 
the  environment,  such  as  the  soil.  Here,  the  soil  is  considered  an  admittance,  and  the 
excavator  boom  as  having  an  impedance  which  can  be  modified  by  the  controller. 

Path  planning,  which  is  responsible  for  identifying  the  most  efficient  means  to  fill 
the  bucket  with  soil,  requires  data  and  information  for  intelligent  decision  making. 
Research  in  robotic  excavation  has  shown  that  forces  and  positions  measured  during 
excavation  reveal  the  "grammar"  describing  the  soil  conditions  much  like  a cone- 
penetrometer.  The  analysis  of  actual  data  provided  the  basis  for  describing  the  mechanics 
of  an  automatic  path  optimization  system  which  would  be  able  to  adapt  automatically  to 
different  bucket  configurations,  task  objectives,  and  soil  characteristics. 

Adaptive  control  models  have  also  been  tested  for  other  applications  such  as 
robotic  masonry,  and  automated  rebar  bending. 
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EVOLUTIONARY  DEVELOPMENT 


OF  TOOLS  AND  MACHINES 


Streamlined  M&D  Flow. 


Modified 

Mechanical 
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REVOLUTIONARY  DEVELOPMENT 
OF  TOOLS  AND  MACHINES 
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IMPEDANCE  CONTROL  MODEL 
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SMALL  SCALED  EXPERIMENTAL  WORK 
ON  PATTERN  RECOGNITION 
FOR  "SMART"  EXCAVATION 
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DATA  INTERPRETATION  WITH 
PATTERN  RECOGNITION 
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ROBOTIC  EXCAVATION 
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CONTROL  AND  INFORMATION  ISSUES  FOR  OPERATING  LARGE 
MANIPULATORS  AND  INTEGRATED  EQUIPMENT  “WORKCELLS”  IN 

TELEOPERATION  MODE 

by 

Dr.  Antal  K.  Bejczy,  Jet  Propulsion  Laboratory 
California  Institute  of  Technology,  Pasadena,  CA  91109 


The  on-line  operator  interaction  with  the  remote  system  is  a combination  of 
perceptive,  cognitive,  decision  making,  and  action  taking  elements.  The  actions  to  be 
taken  are  typically  manual  ones.  For  a successful  teleoperated  system  design  and  for 
productive  system  operation  it  is  imperative  to  look  at  the  control  and  information  issues 
not  only  from  the  viewpoint  of  the  equipment  but  also  from  the  viewpoint  of  the  human 
operators. 

The  information  issues  can  be  grouped  into  three  technical  areas:  sensing  or  data 
acquisition  at  the  work  site,  data  transmission  from  the  remote  work  site  to  the  control 
station,  and  information  displays  to  the  operator  in  the  control  station.  The  sensing  of  the 
remotely  operated  machine’s  internal  state  is  an  elementary  requirement,  and  the  operator 
should  be  aware  of  the  machine’s  internal  state  data  in  the  control  station.  The  other 
sensing  domain  is  the  machine’s  external  state  relative  to  the  work  or  work  environment. 
Here,  the  visual  sensors  (TV  cameras),  including  their  placement  and  control,  play  a key 
role,  since  remote  control  to  a large  extent  is  a visual  perception  problem  to  the  operator. 
Proximity  or  near-contact  sensors  can  add  a necessary  supplement  to  the  TV  camera 
information  since  near-contact  events  in  many  cases  are  not  available  from  TV  camera 
images.  Proximity  sensors  can  also  add  an  automatic  protective  and  safety  feature  to  the 
remotely  controlled  operation.  Another  important  external  state  sensing  registers  the 
machine’s  interaction  force  or  torque  with  die  environment.  This  sensing  can  also  be 
used  to  provide  a scaled  force  or  torque  feedback  to  the  operator’s  hand  through  a proper 
hand-controller  device.  Sound  sensors  (microphones)  can,  in  many  cases,  convey 
important  bits  of  audible  information  to  the  operator’s  always  open  and  omnidirectional 
audio  channel.  The  state-of-the-art  in  the  above  discussed  sensing  devices  is  quite 
advanced.  However,  the  adaptation  of  the  sensors  and  their  instrumentation  to  the 
machine  feature  and  to  the  work  environment  may  require  development  efforts. 

The  sensor  data  transmission  from  the  remote  work  site  to  the  control  station  can 
present  design  decision  issues:  tethered  or  wireless  transmission.  Some  application  cases 
may  permit  feasible  tethered  data  transmission.  The  bulk  of  the  highway  application 
cases,  however,  may  require  wireless  data  transmission.  The  technology  does  exist  for 
that,  but  its  adaptation  to  the  application  environment  will  require  development  work  and 
testing. 


A main  issue  is  the  way  and  method  by  which  the  information  is  presented  to  the 
operator  in  a well-organized  manner,  taking  also  account  of  human  factors.  Today’s 
computer  graphics  technology  offers  very  useful  tools  to  deal  with  information 
organization  and  display  issues  in  a teleoperator  control  station.  In  addition  to  that,  use  of 
today’s  computer  graphics  technology  permits  the  creation  of  a “virtual  reality”  in  the 
following  sense:  we  can  build  high-fidelity  graphics  models  of  working  machines  and  of 
objects  of  interest,  and  these  graphics  models  can,  with  high  fidelity,  be  calibrated  into  a 
given  TV  image  frame  which  covers  the  sight  of  the  machine  and  objects  of  interest. 

After  TV  camera  calibration  and  object  localization,  the  graphics  images  of  machines  and 
objects  of  interest  can,  with  high  fidelity,  be  overlaid  over  the  actual  images  of  machines 


and  objects  of  interest  in  a given  TV  camera  image  frame.  This  creates  a virtual  (or 
phantom)  representation  of  machines  and  objects  in  a given  working  environment  as  seen 
by  the  operator  on  a TV  camera  monitor.  These  types  of  “virtual  reality”  graphics 
displays  are  very  useful  tools  for  planning  and  previewing  controlled  actions  by 
controlling  the  motion  of  graphics  images  first  before  controlling  the  motion  of  the  real 
machine.  These  “virtual  reality”  displays  can  also  be  used  for  operator  training  and 
action  monitoring.  An  interesting  property  of  these  calibrated  graphics  displays  is  the 
capability  of  providing  “synthetic  TV  camera  views”  to  the  operator.  These  views  enable 
the  operator  to  see  events  which  are  not  visible  within  a given  real  TV  camera  view  or  to 
see  events  for  which  no  real  TV  camera  view  is  available. 

The  control  issues  can  be  summarized  under  the  three  categories:  types  of  control 
(position,  rate,  force),  modes  of  control  (manual,  computer,  combined  manual  and 
computer),  and  vontrol  system  architectures.  All  three  control  issue  categories  have  an 
existing  broad  technology  base,  though  a varying  levels  of  maturity  in  the  three 
categories.  Therefore,  a few  remarks  are  in  place  here. 

Control  of  large  manipulators  typically  implies  the  control  of  not  fully  rigid  but 
somewhat  flexing  mechanisms.  Computer  control  typically  implies  the  automation  of 
some  manipulator  motions  or  actions.  Combined  or  shared  computer  (automatic)  and 
manual  control  typically  implies  that  the  operator’s  manual  commands  are  guided  by  the 
operator’s  visual  perception  in  some  motion  direction,  while  at  the  same  time  in  some 
other  directions  automatic  computer  controls  are  used  referenced  to  some  sensor 
information.  A typical  example  is  an  insertion  task  under  visually  narrow  tolerances, 
where  the  forward  insertion  motion  can  be  manually  controlled  under  the  operator’s 
visual  guidance,  and  the  lateral  displacement  and  misalignment  errors  during  insertion 
can  be  automatically  controlled  referenced  to  force-torque  sensor  data  to  prevent 
jamming.  If  we  try  to  put  all  these  control  ingredients  together  into  an  actual  applied  and 
remotely  operated  system  (large  flexing  arm,  shared  manual  and  sensor  referenced 
computer  control),  the  system  may  not  be  fully  ready  from  an  operator’s  viewpoint  for 
actual  use.  It  will  need  experiments,  tests  and  verification,  and  operator  training. 

In  conclusion,  then,  it  seems  worthy  to  seriously  consider  the  development  of  a 
realistic  demonstration  system  to  adjust  and  verify  technology  for  practical  applications, 
including  the  experience  of  the  operators  in  the  control  station. 


LARGE  VOLUME  MANIPULATION  - TECHNOLOGY  OPPORTUNITIES  AND 
LESSONS  LEARNED  FROM  THE  APPLICATION  OF  REMOTE 
MANIPULATION  SYSTEMS  IN  NUCLEAR  ENVIRONMENT 

by 

Dr.  H.  Lee,  President,  TeleRobotics  International,  Inc. 

Manipulation  of  tasks  via  total  or  selective  autonomy  poses  significant  challenges 
for  the  technical  and  economic  implementation  in  real-world  applications.  In  many 
forms,  the  nuclear  industry  has  implemented  remote  manipulation  for  maintenance  and 
decommissioning  activities.  Since  construction  faces  similar  challenges  (e.g.,  heavy 
loads  to  be  manipulated)  tested  concepts  established  in  the  areas  of  fundamental  design, 
system  integration,  performance,  and  reliability  related  to  the  successful  implementation 
of  manipulation  systems  in  unconstrained  environments.  In  addition,  the  increased 
complexity  of  the  tasks  provides  fosters  many  emerging  technologies  for  manipulation 
and  sensing. 
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DESIGN  OF  HUMAN-MACHINE  INTERFACES  IN  NEW  HIGHWAY 
CONSTRUCTION  AND  MAINTENANCE  EQUIPMENT 

by 

H.  Mcllvaine  Parsons 

Manager,  Center  for  Human  Factors,  Human  Resources  Research  Organization 

(HumRRO),  Alexandria,  Virginia 

The  following  statements  focus  more  on  the  development  as  a driver  of  research 
than  on  human  factors/ergonomic  research  itself.  A few  years  ago,  in  a project  for  the 
Army’s  Human  Engineering  Laboratory,  a survey  was  conducted  to  assess  design 
aspects  related  to  human-machine  interfaces  in  remote  control  centers  and  test  beds  for 
teleoperated  vehicles(Parsons,  1989).  Though  the  total  of  32  organizations  studied  was 
not  exhaustive,  it  showed  the  extent  to  which  such  interfaces  were  already  proliferating; 
the  number  today  is  surely  much  greater,  with  mobile  teleoperators  installed  in  ground 
(interior  and  exterior),  underwater,  air,  and  space  environments  or  being  planned  for 
these.  Problems  with  operator  interface  have  always  existed,  however,  it  is  receiving 
increasing  attention  from  the  science  and  engineering  community  concerned  with  robotics 
and  teleoperation. 

The  above  mentioned  survey  covered  the  entire  range  of  pertinent  design  aspects: 
vehicle  and  manipulator  characteristics,  sensors,  personnel  positions, 
configuration/layout,  reconfigurability  and  adjustability,  control  devices  (hand-,  finger-, 
foot-,  and  voice-operated),  display  elements  (CCTV  monitors,  computer-associated 
screens,  other  displays,  special  displays/inputs  to  the  operator),  audio  elements,  and 
communications.  All  of  these  varied  considerably  among  control  centers  and  test  beds, 
suggesting  some  uncertainty  about  what  was  superior  as  well  as  what  might  need 
improvement  through  applied  research.  Although  much  development  and  installation  can 
now  proceed  with  off-the-shelf  components,  each  new  system  requires  systematic  study 
of  its  particular  requirements. 

One  should  also  consider  the  human-machine  interface  in  industrial  robotics 
(Parsons,  1988, 1992).  In  addition  to  off-line  programming,  the  principal  interface  device 
is  a remote  control  unit  called  a teach  pendant.  In  on-line  programming  this  controls  a 
manipulator  much  as  though  it  were  being  teleoperated.  Also  for  the  Army’s  Human 
Engineering  Laboratory  I surveyed  the  designs  of  teach  pendants  used  by  ten  major  U.S. 
and  Japanese  robot  manufacturers.  The  designs  of  these  have  differed  so  greatly  that  the 
Robotic  Industries  Association,  with  my  participation  and  help  from  pendant  engineers, 
and  drawing  both  on  my  survey  and  on  MilStd  1472C,  developed  ANSI  standard 
ANSI/RIA  R15.02/1-1990  for  Industrial  Robots  and  Robot  Systems-Hand-Held  Robot 
Control  Pendants— Human  Engineering  Design  Criteria. 

A study  sponsored  by  the  National  Institute  of  Aging  investigated  how  often  older 
people  might  make  mistakes  simply  in  pressing  pushbuttons.  The  error  frequencies  of 
most  of  the  18  participants  65  to  88  years  old  in  an  observational  study  raised  some 
doubts  about  the  feasibility  of  using  teleoperators  for  household  tasks,  even  if  interface 
designs  were  optimized. 

Design  guidelines  and  simple  prototype  testing  can  be  helpful  in  developing 
human-operator  interfaces,  along  with  task  analyses,  workload  analysis,  review  of  end- 
user  characteristics,  and  study  of  safety  considerations  and  environmental  conditions. 

Any  operational  testing  of  equipment  and  software  should  embody  human  factors 
components,  including  representative  future  operators,  maintainers,  and  programmers. 


Indeed,  at  the  start  of  system  development  such  eventual  end-users  should  be  queried 
methodically  about  their  experience  with  current  equipment,  and  they  should  be 
consulted  as  the  new  system  is  installed. 

References: 

Parsons,  N.M.  (1988).  Robot  programming.  In  M.  Helander  (Ed.),  Handbook  of  human- 
computer  interaction.  Amsterdam:  Elsevier. 

Parsons,  H.M.  (1989).  Teleoperator  interfaces  for  remote  control  centers/test  beds.  In 
Proceedings  of  the  Human  Factors  Society  33rd  Annual  Meeting.  Santa  Monica,  CA: 
Human  Factors  and  Ergonomics  Society. 

Parsons,  H.M.  (1992).  Remote-control  units  for  industrial  robots.  In  M.  Rahimi  and  W. 
Karwowskd  (Eds.),  Human-robot  interaction.  London:  Taylor  & Francis. 
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Technologies  for  Inspection  of  Bridges  and  Road  Surfaces, 
Automated  Surveying,  “As  Built”  Databases,  Site 
Positioning  and  Quality  Assurance 
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Figure 


Figure  3 


Robot  Vision  Lob.  Purdue  University 


Figure  4 


Robot  Vision  Lob.  Purdue  University 


X-scan  Range  Data  Gathering 


Figure  6 


Robot  Vision  Lob.  Purdue  University 


Z as  intensity 


Figure  7 


Robot  Vision  Lob.  Purdue  University 
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Figure  8 


Robot  Vision  Lob.  Purdue  University 


Photometric  Data  Gathering 


Intensity  image 


Figure  10 


Robot  Vision  Lab.  Purdue  University 


Figure  1 1 
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Figure  1 2 


Robot  Vision  Lob.  Purdue  University 


Purdue  Robot  Vision  Lab's  Neuro-Morph  Architecture 
For  Surface  inspection  at  Multiple  Levels  of  Resolution 


Figure  13 


Robot  Vision  Lob.  Purdue  University 


Use  of  Mathematical  Morphology  for  analysis  of  pavement  images 
Presented  by  Michael  M.  Skolnick 
Dept  of  Computer  Science 
R.P.I. 

Troy,  N.Y.  12180 
Abstract 

Image  processing  algorithms  based  on  the  transformations  of  mathematical  morphology  are  applied  to 
the  problems  of  pavement  inspection.  Mathematical  morphology  holds  out  some  promise  to  these 
problems  since  it  was  developed  out  of  the  applications  of  materials  analysis  and  pavement  surfaces 
can  be  considered  as  particle/phase  structures.  The  task  is  to  classify  pavement  images  into  various 
distress  categories,  with  the  major  components  of  distress  involving  the  detection  of  cracks  and 
changes  in  surface  texture.  A pavement  image  is  processed  as  if  it  consists  of  various  kinds  of  particle 
size  distributions.  Such  size  distributions  are  shown  to  be  sensitive  both  to  cracks  and  to  changes  in 
the  expected  texture  of  the  pavement  images.  The  system  can  be  configured  to  have  sensitivity  to  a 
wide  range  of  expected  textures  via  the  use  of  a normalizing  function  applied  to  the  raw  size 
distribution  data.  Also,  due  to  the  reduction  of  image  data  into  particle  size  distribution  data  there  is  a 
significant  computational  advantage  to  this  approach.  Finally,  the  sensitivity  to  phenomena  beyond 
simple  crack  measures  distinguishes  this  approach  from  the  current  state  of  the  art. 
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USE  OF  MATHEMATICAL  MORPHOLOGY 
FOR  ANALYSIS  OF  PAVEMENT  IMAGES 


Rensselaer  Polytechnic  Institute 
Troy,  NY  12180 


Problem  Descripti 


Figure  2.  Pavement  images  belonging  the  class  N - surface  cracking  of  Portland  cement  concrete-  categorized  as  to  three  levels  of  distress. 
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Figure  1.  Pavement  images  belonging  the  class  G - asphault  shoulder  surfaces  - categorized  as  to  four  levels  of  distress. 
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Rough  indication  of  the  location  of  defects  (as  well  as  texture 
inhomogeneities)  is  obtained  from  centroid  movements 
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Analysis  of  Pavement  Images  Normalization  Scliei 
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Analysis  of  Pavement  Images  - Normalization  Scheme 


r 


Rensselaer  Polytechnic  Institute 


Analysis  of  Pavement  Images  Normalization  Scheme 
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Analysis  of  Pavement  Images  Normalization  Scheme 
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ment,  and  subsequently  provide  useful  information  for  project  level 
analysis 


A Neural  Network-Based  Methodology  for  Automated  Detection  and 
Classification  of  Highway  Pavement  Surface  Cracking 


Mohamed  Kaseko  and  Stephen  Ritchie 
Institute  of  Transportation  Studies 
University  of  California 
Irvine,  CA  92717 

One  of  the  most  important  elements  of  an  effective  pavement  management  system 
is  the  collection  and  interpretation  of  pavement  surface  distress  data.  Current 
procedures  for  carrying  out  this  process  typically  involve  on-site  visual 
inspection  and  condition  evaluation  by  field  personnel.  This  method  is  a 
subjective,  slow  process  that  is  also  labor-intensive,  tedious  and  often  dangerous. 

Recent  developments  in  automation  of  this  process  have  principally  been  based  on 
the  application  of  machine  vision  and  conventional  image  processing  techniques. 
Although  these  developments  have  considerably  advanced  the  state-of-the-art  of 
automated  pavement  distress  evaluation,  their  performance  has  been  limited  by 
the  inherent  shortcomings  of  conventional  image  processing  techniques  applied  to 
pavement  images. 

The  objective  of  this  research  was  therefore  to  develop  and  demonstrate  the 
feasibility  of  an  alternative  methodology  that  is  based  on  integration  of 
conventional  image  processing  techniques  and  artificial  neural  network  (ANN) 
models.  The  research  focused  on  application  of  ANN  models  as  pattern 
classifiers  for  image  interpretation  and  classification,  resulting  in  the 
development  of  approaches  for  automatic  thresholding  of  the  images,  and  for 
detection  and  classification  of  surface  cracking  in  each  image,  using  a multi-layer 
feed-forward  (MLF)  neural  network  model.  About  250  of  the  asphalt  concrete 
pavement  images  acquired  by  the  firm  PASCO  USA  INC.  for  the  US  Strategic 
Highway  Research  Program  (SHRP)  were  used  in  this  research. 

The  results  obtained  have  shown  that  MLF  was  able  to  detect  and  correctly 
classify  about  98%  of  the  images  with  transverse  and  longitudinal  cracking,  and 
86%  of  those  with  alligator  and  block  cracking.  A method  for  computation  of 
severity  and  extent  measures  was  also  developed.  These  results  have  clearly 
demonstrated  the  potential  for  application  of  the  neural  network -based  approach 
in  detection  and  classification  of  pavement  surface  cracking. 

This  research  was  part  of  an  effort  aimed  at  developing  an  Al-based  system  that 
would  automate  much  of  the  pavement  data  acquisition,  interpretation,  and 
evaluation  process,  and  capture  the  experience  and  judgment  of  expert  pavement 


engineers  in  performing  condition  assessments  and  identification  of  appropriate 
maintenance  and  rehabilitation  strategies.  One  of  the  basic  elements  of  the  system 
already  exists  in  prototype  form.  It  is  a microcomputer-based  knowledge-based 
expert  system  known  as  Pavement  Rehabilitation  Analysis  and  Design  Mentor 
(PARADIGM),  which  takes  as  input  pavement  distress  and  other  data  and 
performs  condition  assessment  and  recommends  repair,  maintenance  and 
rehabilitation  strategies. 


NSF/NIST/FHWA  Workshop  on  April  28,  1993 
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